Nostr notes by René Pickhardt [ARCHIVE]

📅 Original date posted:2023-01-01 📝 Original message: Happy ...

2023-06-09T13:07:46Z

In reply to nevent1q…x3el
_________________________

📅 Original date posted:2023-01-01
📝 Original message:
Happy new year dear fellow Lightning Network Developers,

last month I have made a small observation why we probably should at most
progress EITHER with `negative fees` [1] OR `upfront fees` [2] but not with
BOTH as adding both features to the protocol would result in a potentially
lucrative attack that I will describe here.

Assumption:
=========

For simplicity of the argument please assume all nodes do payment delivery
by optimizing purely for fees instead of probabilistic payment delivery or
a combination of these two and potentially other features in their cost
function. The Argument will however work as long as fees are part of the
cost function.

The Attack:
=========

1. Malory sets the routing fees of her channel(s) sufficiently negative.
2. Now the cheapest route for all possible payment pairs on the network
goes through Malory.
3. Malory will accept any incoming HTLC but will shortly after the HTLC is
locked in fail the payment without forwarding.
( 4. Depending on the design of upfront fees she may need a collaborating
proxy node)

Outcome:
========

1. After announcing the negative routing fees every node that has seen the
`channel_update` will route through Malory if initiating a payment. This
effectively redirects the entire traffic of the network through her node.
2. Malory has create a DoS attack on her own node but depending on the size
of the network she will not even see it as her channel partners will go
down from the DoS first (or she is able to handle the traffic as she was
prepared)
3. Assuming Malory has enough Channel partners (or collaborates with them)
she can collect the tiny unconditional upfront fees (Depending on the price
of the upfront fees, the size of the network and the base load of payments
per second this may or may not be lucrative)

Also as her fees were so negative most nodes might not even blame her for
the routing failures as they might assume others were just more quickly
sniping that juicy liquidity. Yet Alice has collected some upfront fees of
all payments that are going on at that time.

Some thoughts about mitigation strategies:
=================================

## Working:
* Choose weather to progress with either negative fees or upfront fees will
stop this particular problem to come up.

## Probably not working:
* Forcing channels with negative fees to set the the upfront fee negative
will not work. This is effectively handing out free money to the channel
partner: As soon as someone announces negative fees the channel partner
will send out fake payments and earn the negative upfront fee.
* Allow `channel_updates` only to be relayed from connections that maintain
a channel so that Mallory cannot quickly inform the entire network about
being the most central node by connecting to everyone may help in
combination with rate limiting of payments and reputation ideas but I guess
others are more experienced than me with reputation systems. Also I think
new participants need `channel_updates` even if they don't have channels
yet.

Own thoughts:
===========

As many of you know I am currently writing a paper about the fundamental
limitations of the scaling abilities of the Lightning Network to conduct
Bitcoin payments [3]. Most folks I talk to see deliberate and malicious
channel jamming as a problem. While I agree with the problem I think the
situation is worse. It is my current understanding that natural congestion
resulting from the selfish behavior of both sending and routing nodes will
be a huge challenge for the network. This is amplified by the uncertainty
(for example about liquidity). However, even without uncertainty it will
create an upper boundary of how many payments per second the participants
of the network will be able to conduct. This boundary is more or less given
by the weighted betweenness centrality of the most central node and the
routing throughput that this node is able to handle. More on this is soon
to come here...

That being said, independently of the up-front fees it seems to me that
allowing negative fees tend to increase centralization effects and thus the
price of anarchy and natural congestion. Yet I can't quantify this at this
time and thus I don't know yet if this fundamentally speaks against
negative fees. However as discussed above in combination with upfront fees
there seems to be an economic incentive to abuse both together.

Thanks to Christian Decker for spotting an error in an edge cases when I
initially presented him a similar argument for review.

with kind regards Rene Pickhardt

[1]
https://lists.linuxfoundation.org/pipermail/lightning-dev/2022-September/003685.html
[2]
https://lists.linuxfoundation.org/pipermail/lightning-dev/2022-November/003740.html
[3] https://twitter.com/renepickhardt/status/1605189724293169153
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20230101/7e0f200b/attachment.html>;

📅 Original date posted:2022-09-23 📝 Original message: Dear ...

2023-06-09T13:06:51Z

In reply to nevent1q…3xj5
_________________________

📅 Original date posted:2022-09-23
📝 Original message:
Dear Matt and Lightning developers,

those are excellent and important questions that I should probably have
addressed more explicitly in the article / ml-post! Let me add what I
currently know and begin with your second question as I think I think the
answer will be more objective / verifiable while my response to the gossip
question is a bit more speculative at this point:

On Fri, Sep 23, 2022 at 10:43 AM Matt Corallo <lf-lists at mattcorallo.com>
wrote:

b) What are the privacy implications of the naive "update on drained
> channel", and have you done any
> analysis of the value of this type of gossip update at different levels of
> privacy?
>

We know from information theory that the distribution with the highest
entropy is the uniform distribution [1] which in the context of a channel
with capacity of `c` means `log(c+1)` bits as shown in [2].

Now if a given drain leads to a pair of `htlc_maximum_msat` one could in
deed also go the other way around and look at the pair of
`htlc_maximum_msat` and estimate the *past* drain. And of course also the
(the non uniform) liquidity distribution from the past. Note that now with
the better `htlc_maximum_msat` pair the distribution should be closer to
uniform again which is actually best from an information theoretic point of
view as this maximizes the entropy in the channel.

I have just created and uploaded a small python script / notebook [3] from
which we can see for example how much information about the shape of the
past liquidity distribution we would learn if we could induce from the
`htlc_maximum_msat` pair that the drain was 0.75. So from the notebook I
quote:

Assume we learnt a drain of 0.75
=================================
Entropy of uniform: 20.00 bits.
Entropy of the channel with drain: 19.17 bits
Information gain: 0.83 bits

Yes we learnt less than 1 bit of information by knowing the shape of the
distribution from the last payments. This much is learnt by a single probe.
Given that we expect our payment failure rates to drop by an order of
magnitude and the fact that I could probably learn the drain from the on
chain signal or gossip anyway I personally would consider this information
leakage to be acceptable. But of course people may not agree with me and
just not set their `htlc_maximum_msat` flag in the proposed manner. Also it
is obvious that higher drain values produce a more heavily depleted channel
and we might learn more bits (in case of a 0.95 drain we learnt about 3
bits.)

a) How much gossip overhead do you expect this type of protocol to
> generate/is there a useful
> outcome for this type of update even if you limit gossip updates to
> once/twice/thrice per day?
>

As written in the article I expect the setting of the valves as throtteling
devices to be rather stable as I would assume the drain on channels should
not be impacted too heavily by installing valves. This assumption is
certainly more reasonable if nodes already used min cost flow solvers to
send out payments and less reasonable if nodes still use ad-hoc splitting
heuristics and restrict their path finding on channels with sufficiently
large `htlc_maximum_msat` values for the amount of the partial payment.

In any case I expect that once the valve is in a good setting for a
particular channel / drain the setting can stay as long as the drain won't
change significantly. In general I would not expect drain on channels to
change heavily over time (though I certainly would not expect it to be
static / stable either). Certainly fee changes in the network (and other
factors) may eventually produce changes in drain on channels which would
also have to eventually be reflected in new settings for the valve.

The experimental algorithm idea that I presented in cell 12 of the original
notebook (which I also used in the privacy consideration code from above)
computes the histogram of liquidity of the last N routing requests (failed
and successful ones) that a node saw on the channel and then makes a
decision weather to open or close the valve a bit more. Using this greedy
strategy we saw in the notebook that a node could find decent values in
logarithmic many steps . So while the setup of the valve may trigger a few
gossip messages (but each one only after a sufficient amount of routing
requests have been processed) and while I expect dynamics in drain on the
channel I don't expect such adoptions should have to happen in real time or
even several times per day and put unnecessary load / spam to gossip. But I
guess we can only see this by operating nodes and probably there are some
nodes where updates occure more frequently than with other nodes.

That being said of course if the peers of a channel work together and both
exchange information / collaborate in finding a good `htlc_maximum_msat`
pair (similalry as our protocol for finding fees in a mutual close) they
will probably need less gossip messages than in the case where both nodes
independently try to find decent settings.

I am happy if people have more insights into this and challenge my
expectation (because it actually really is only an expectation / intuition
at this point). That being said: Yes even with only a few gossip messages
per day I expect for the given reasons the setup of valves to be possible
and useful!

with kind regards Rene

[1]:
https://en.wikipedia.org/wiki/Maximum_entropy_probability_distribution#Uniform_and_piecewise_uniform_distributions
[2]: https://arxiv.org/abs/2103.08576
[3]:
https://github.com/lnresearch/Flow-Control-on-Lightning-Network-Channels-with-Drain-via-Control-Valves/blob/main/Privacy%20Considerations%20of%20signaling%20past%20drain%20via%20%60htlc_maximum_msat%60%20pairs.ipynb

>
> Thanks,
> Matt
>
> On 9/22/22 2:40 AM, René Pickhardt via Lightning-dev wrote:
> > Good morning fellow Lightning Developers,
> >
> > I am pleased to share my most recent research results [1] with you. They
> may (if at all) only have a
> > small impact on protocol development / specification but are actually
> mainly of concern to node
> > operators and LSPs. I still thought they may be relevant for the list.
> >
> > While trying to estimate the expected liquidity distribution in depleted
> channels due to drain via
> > Markov Models I realized that we can exploit the `htlc_maxium_msat`
> setting to act as a control
> > valve and regulate the "pressure" coming from the drain and mitigate the
> depletion of channels. Such
> > ideas are btw not novel at all and heavily used in fluid networks [2].
> Thus it seems very natural
> > that we do the same on the Lightning Network.
> >
> > In the article we show within a theoretic model how expected payment
> failure rates per channel may
> > drop significantly by up to an order of magnitude if channels set up
> proper asymmetric
> > `htlc_maximum_msat` pairs.
> >
> > We furthermore provide in our iPython notebook [3] two experimental
> algorithmic ideas with which
> > node operators can find decent `htlc_maximum_msat` values in a greedy
> fashion. One of the algorithms
> > does not even require to know the drain or payment size distribution or
> build the Markov model but
> > just looks at the liquidity distribution in the channel at the last x
> routing attempts and adjusts
> > the `htlc_maximum_msat` value if the distribution is to far away from a
> uniform distribution.
> >
> > Looking forwards for your thoughts and feedback.
> >
> > with kind regards Rene
> >
> >
> > [1]:
> >
> https://blog.bitmex.com/the-power-of-htlc_maximum_msat-as-a-control-valve-for-better-flow-control-improved-reliability-and-lower-expected-payment-failure-rates-on-the-lightning-network/
> <
> https://blog.bitmex.com/the-power-of-htlc_maximum_msat-as-a-control-valve-for-better-flow-control-improved-reliability-and-lower-expected-payment-failure-rates-on-the-lightning-network/
> >
> > [2]: https://en.wikipedia.org/wiki/Control_valve <
> https://en.wikipedia.org/wiki/Control_valve>;
> > [3]:
> >
> https://github.com/lnresearch/Flow-Control-on-Lightning-Network-Channels-with-Drain-via-Control-Valves/blob/main/htlc_maximum_msat%20as%20a%20valve%20for%20flow%20control%20on%20the%20Lightnig%20network.ipynb
> <
> https://github.com/lnresearch/Flow-Control-on-Lightning-Network-Channels-with-Drain-via-Control-Valves/blob/main/htlc_maximum_msat%20as%20a%20valve%20for%20flow%20control%20on%20the%20Lightnig%20network.ipynb
> >
> >
> > --
> > https://ln.rene-pickhardt.de <https://ln.rene-pickhardt.de>;
> >
> > _______________________________________________
> > Lightning-dev mailing list
> > Lightning-dev at lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

--
https://www.rene-pickhardt.de
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220923/b0791f3f/attachment-0001.html>;

📅 Original date posted:2022-09-22 📝 Original message: Good ...

2023-06-09T13:06:51Z

In reply to nevent1q…a5s9
_________________________

📅 Original date posted:2022-09-22
📝 Original message:
Good morning fellow Lightning Developers,

I am pleased to share my most recent research results [1] with you. They
may (if at all) only have a small impact on protocol development /
specification but are actually mainly of concern to node operators and
LSPs. I still thought they may be relevant for the list.

While trying to estimate the expected liquidity distribution in depleted
channels due to drain via Markov Models I realized that we can exploit the
`htlc_maxium_msat` setting to act as a control valve and regulate the
"pressure" coming from the drain and mitigate the depletion of channels.
Such ideas are btw not novel at all and heavily used in fluid networks [2].
Thus it seems very natural that we do the same on the Lightning Network.

In the article we show within a theoretic model how expected payment
failure rates per channel may drop significantly by up to an order of
magnitude if channels set up proper asymmetric `htlc_maximum_msat` pairs.

We furthermore provide in our iPython notebook [3] two experimental
algorithmic ideas with which node operators can find decent
`htlc_maximum_msat` values in a greedy fashion. One of the algorithms does
not even require to know the drain or payment size distribution or build
the Markov model but just looks at the liquidity distribution in the
channel at the last x routing attempts and adjusts the `htlc_maximum_msat`
value if the distribution is to far away from a uniform distribution.

Looking forwards for your thoughts and feedback.

with kind regards Rene

[1]:
https://blog.bitmex.com/the-power-of-htlc_maximum_msat-as-a-control-valve-for-better-flow-control-improved-reliability-and-lower-expected-payment-failure-rates-on-the-lightning-network/
[2]: https://en.wikipedia.org/wiki/Control_valve
[3]:
https://github.com/lnresearch/Flow-Control-on-Lightning-Network-Channels-with-Drain-via-Control-Valves/blob/main/htlc_maximum_msat%20as%20a%20valve%20for%20flow%20control%20on%20the%20Lightnig%20network.ipynb

--
https://ln.rene-pickhardt.de
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220922/a895d1a1/attachment.html>;

📅 Original date posted:2022-08-25 📝 Original message: Dear ...

2023-06-09T13:06:43Z

In reply to nevent1q…ylgw
_________________________

📅 Original date posted:2022-08-25
📝 Original message:
Dear fellow Lightning Developers,

I was recently on an event where the visitors have been gifted 10k sats on
a custodial wallet. They could spend those sats via some web interface and
an NFC card. During the event I was contacted by several plebs who were
confused about one particular thing:

It was impossible for them to withdraw the full amount from the service.

Pasting an invoice for 10k sats would not work as the custodial service
required a fee budget of 1%. However if people submitted an invoice for
9900 sats the remaining 100 sats were usually not fully required for the
fees. Thus the users may have had a leftover of for example 67 sats. Now
the problem repeated on the residual amount. While some services seem to
have a drain feature for such a situation I find this frustrating and was
wondering if we could help directly on a protocol level.

Here is my proposal for a simple solution to this specific problem:
`option_recipient_pays_routing_fees`

This would be a new flag in invoices signaling that the recipient is
willing to pay for the routing fees by releasing the preimage even if the
full amount has not been arrived in htlcs at the recipient.

So the workflow would be the following:

1. Alice creates an invoice for 10k sats setting the
`option_recipient_pays_routing_fees` flag in the invoice and passes it
either to custodial user Bob or to her own custodial account.
2. The payer parses the invoice and searches for a payment path or payment
flow to Alice.
3. Because `option_recipient_pays_routing_fee` is set, the onion is not
constructed in a way that the final HTLC will be for the amount of 10k sats
but rather in a way that the first htlc will be for 10k sats and the
following HTLCs will be of decreasing value so that routing nodes are
compensated properly.
4. When the HTLC(s) arrive at Alice she will release the preimage if and
only if not too many sats (e.g. 1% of the amount) are missing. Of course it
would be good if the 1% was not hard coded in the protocol / software but
configurable by Alice at the time of invoice creation.

I think the main issue with this proposal is that instead of confusing
users who wish to drain an account we may now have to educate users about
two different invoice types. On the other hand I think this can probably
easily be achieved via the current wide spread user interfaces. Of course
it may be nice to have folks from the Bitcoin Design community to join this
specific part of the discussion.

With kind regards Rene Pickhardt
--
https://www.rene-pickhardt.de
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220825/400b3ecc/attachment.html>;

📅 Original date posted:2022-06-08 📝 Original message: Dear ...

2023-06-09T13:06:12Z

In reply to nevent1q…k6mf
_________________________

📅 Original date posted:2022-06-08
📝 Original message:
Dear Tony,

Thank you for putting emphasis on this. I was actually waiting for someone
to publicly exploit this.

> The reason this is possible is because [...] currently channel IDs are
> based on UTXO's. Scid aliases may be the biggest benefit here, but the use
> of `unknown_next_peer` , `invalid_onion_hmac`, `incorrect_cltv_expiry`,
> and `amount_below_minimum` have been the biggest helpers in exploiting
> channel privacy.
>

Just for reference the exploit with short_channel_ids is known since 2019:

https://github.com/lightning/bolts/issues/675

Though it is nice you point out explicitly the use of error codes of
onions.

> By creating a probe guessing the Channel ID based on unspent p2wsh
> transactions, it's a `m * n` problem to probe the entire network, where `m`
> is utxos and `n` is nodes.
>

It is the main reason why I didn't do this. Though similar to you probing
ACINQ's node one could probabilistically learn which nodes tend to have
unannounced channels and gain some speedup by probing those nodes first.

Also wallets tend to have poor utxo management. So looking at the on-chain
signal one can probably guess for a p2wsh to which two nodes it might
belong and try them first.

These two strategies should reduce the number of tested nodes for a newly
seen p2wsh output significantly and probably make it feasible to probe the
network as new blocks come in.

With kind regards Rene Pickhardt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220608/08a200ad/attachment.html>;

📅 Original date posted:2022-03-14 📝 Original message: Dear ...

2023-06-09T13:05:31Z

In reply to nevent1q…8eas
_________________________

📅 Original date posted:2022-03-14
📝 Original message:
Dear Carsten, Martin and fellow lightning developers,

first of all thank you very much for independently verifying and
acknowledging my recent findings about the runtime of finding a pieceweise
linearized approximation to the min cost flow problem, for working on
integrating them into lnd-manageJ and for your excellent questions &
thoughts.

On Sun, Mar 13, 2022 at 8:17 PM Carsten Otto via Lightning-dev <
lightning-dev at lists.linuxfoundation.org> wrote:

> 1) What's the reasoning behind combining parallel channels?
>

Generally speaking this is pure pragmatism on my end to simplify my life as
handling parallel channels in some cases blows up complexity of code and
simulations. However I think from a probabilistic point of view ( see below
) the combination is more accurate to reflect the actual likelihood that
the liquidity is available.

I agree that parallel channels make things a lot more complicated, but I
> also see the benefit from a node operator's point of view. That being
> said, wouldn't it suffice to treat parallel channels individually?
>

I think that should work and especially when including fees to the cost
function and considering how nodes handle routing requests on parallel
channels we might have to do so anyway. The suggested flows will probably
change in a way that disfavors parallel channels even if their virtual
capacity is larger than an alternative single channel (see below)

1.1) A payment of size 2 needs to be split into 1+1 to fit through
> parallel channels of size 1+1. Combining the 1+1 channels into a virtual
> channel of size 2 only complicates the code that has to do come up with
> a MPP that doesn't over-saturate the actual channels. On the other hand,
> I don't think the probability for the virtual channel of size 2 is more
> realistic than reasoning about two individual channels and their
> probabilities - but I didn't even try to see the math behind that.
> Please prove me wrong? :)
>

* The likelihood that a 1 Satoshi capacity channel has 1 Satoshi to route
is 1/2.
* The likelihood that 2 channels of capacity 1 have each 1 satoshi
available to route is 1/2*1/2 = 1/4
* Combining both parallel channels to one virtual channel of capacity 2 and
asking if 2 satoshis are available to route gives a likelihood of 1/3 which
is larger than 1/4.

However I believe in practice one cannot just send a 2 satoshi onion and
expect the routing node to split the amount correctly / accordingly
between the two parallel channels. (I might be wrong here). So in that case
modelling and computing probabilities for parallel channels might be
necessary anyway though the math indicates that splitting liquidity in
parallel channels will get you selected less frequently for routing.

1.2) The Mission Control information provided by lnd can be used to
> place a minimum available balance on each of the parallel channels. If
> we know that node A isn't able to forward N sats to node B, we can treat
> all parallel channels between A and B (in that direction) to have a
> capacity of at most N-1 sats. How would this look like if we combined
> the parallel channels into a virtual one? Note that it may still be
> possible to route two individual payments/onions of size N-1 sats from A
> to B, given two parallel channels with that many sats on A's side.
>

I think you talk a about a maximum available balance of a channel (and not
min available balance)?
In the case of parallel channels I am not even sure if such information is
accurate as it is my understanding that the routing node may decide to use
the parallel channel to forward the amount even though the other channel
was specified in the onion.
Assuming that routing nodes indeed do so we would have learnt that neither
channel has an effective capacity of N. So the combined virtual channel
could be seen as 2N-1. However if routing nodes don't locally split a
forwarding request across both channels we would know that calaculating
with 2N-1 is bad as a request of N could not be fulfilled. I guess it is
for the implementations that support parallel channels to figure out the
details here.

2) Optimal Piecewise Linearization
>
> See Twitter [3].
>
> Is it worth it cutting a channel into pieces of different sizes, instead
> of just having (as per your example) 5 pieces of the same size? If it
> makes a noticeable difference, adding some complexity to the code might
> be worth it.
>

I will certainly do experiments or be happy if others are faster to do them
which compare the quality of the approximation with optimal piecewise
linearization to my choice of fixed intervals and the selection of various
numbers of segments. As long as we don't have numbers it is hard to guess
if it is worthwhile adding the complexity. Looking at the current results
it seems that my (geometricly motivated but) arbitrary choice might end up
to be good and easy enough. However we might very well see quite an
improvement of the approximation if we find better piecewise linearizations.

> 3) Size of Piecewise Linearization
>
> My gut feeling is that cutting a 1 BTC channel into 5 pieces is
> different from cutting a 0.01 BTC channel into 5 pieces. Would it make
> sense to use different values of N depending on the channel size?
>

The main difference here is that a channel of 1 BTC is highly preferable
from a probabilistic payment delivery perspective over a channel of 0.01
BTC. Even approximating the 1 BTC channel with 1000 intervalls of 0.001 BTC
should still have a lower unit cost in all pieces of the first 0.01 BTC of
the liquidity than the first piece of the 0.01 BTC channel. So I think
splitting all channels in the equal number of pieces is pretty well
motivated but let me elaborate on this:

The motivation of splitting all channels into the same number of pieces
comes from the observation that from a probabilistic point of view (and
very roughly speaking!) we want to find a flow that puts the same (high)
success probability on all edges. Using 20% of the capacity gives an 80%
probability that the liquidity is available. This in turn has a 51.2%
chance to be successful on a three hop path which in my experience is a
good probability to aim for as on average one is expected to need two
attempts. Since the linearized pieces - if included in the flow - tend to
be fully saturated I decided that it makes sense to put them in 5 buckets
of 20% each. If you read the code carefully I don't even approximate the
cost correctly at the 2nd, 3rd, 4th and 5th piece. I just multiplied the
linearized unit cost of the first piece with 2,3,4 and 5 respectively Which
approximates the negative log probability with a quadratic cost function.
But as we can see those geometrically motivated choices work already pretty
well. Again I plan to invest more time to find a better approximation and
we might end up using it. But I don't expect too much gain from doing so.

If you look at the output of the flow in the iPython notebook (Which I
will copy to the end of the mail for your convenience) you see that most
channels did not fully saturate the first piece and have a likelihood
between 80% and 100% where as some channels saturated the first piece
producing a likelihood of 80% and few channels saturated also the second
piece giving a likelihood of 60%.

Thus I expect that the real value from studying the optimal piece wise
linear approximation will give us a better understanding of where to prune
segments away. If we note that in this flow not a single channel used the
third, forth and fifth piece we could have removed 60% of all edges and
doubled the runtime and still compute the same resulting flow.

> 4) Leftovers after Piecewise Linearization
>
> If I cut some channel into N pieces, I might end up with up to N-1 sats
> that don't end up in any of the N pieces, effectively making the channel
> look smaller than it is. For smaller values of N that's obviously not an
> issue (given the uncertainty we're dealing with), but it might be more
> problematic if quantization is used with larger values. Any thoughts on
> this?
>

I am not sure if I understand your question / issue here. The splitting
works by selecting N points on the domain of the function and splitting the
domain into segments at those points. This should never leave sats over.
The quantization which doesn't boost the runtime too much anyway happens
before piecewise linearization. So as long as the domain is larger than N
the piecewiese linearization should not bring up a situation where sats are
left over. If the quantization however makes a channel so small that we
cannot even create 5 (or N) disjoint segments then I guess the likelihood
for being included into the final result is too small anyway. But I agree
that an actual implementation might have to watch out for such edge cases.

Again this yield interesting pruning opportunities to reduce the seize of
the network before doing the expensive min cost flow computation. For
example I could prune channels with high unit costs on the first segment.
Especially if they are further away from the source and destination node.
This would overall reduce the size of the graph and improve runtime.
Already last July I had a pretty well working heurist that was able to
through away about 90% of all channels and would pretty much always find
the same flow in the pruned network as on the full network. I should really
prioritize the pruning work again now that we have fast solvers.

5) Fees (and other properties?)
>
> How can we integrate fees into the function? I must admit, I haven't
> even thought about that, yet. A copy-paste answer would be great,
> though! :) Maybe it's also a good idea to punish channels based on their
> CLTV delta? Ratio of enabled channels? Age? Manual punishment score? ...
>

plugging in fees (assuming only channels that set a base fee of zero) is
very easy and straight forward.

Let's recall from the code that the piecewise linearized unit cost is
computed as follows

unit_cost = int(max_cap/cap)for i in range(N):
#arc format is src, dest, capacity, unit_cost
arcs.append((src,dest,int(cap/(N*QUANTIZATION)),(i+1)*unit_cost))

As described in our paper for a good (and to be determined) value of \mu we
could just create the linear combination between the two features which
would change the line to:

arcs.append((src,dest,int(cap/(N*QUANTIZATION)),(i+1)*unit_cost +
mu*fee_rate_ppm))

Note two things:
1. the only requirement for the solver to work is that \mu*fee_rate_ppm
needs to be an integer. So in case \mu was smaller than 1 we could also
scale the term from the linearized log probabilities by putting a larger mu
to the feature arising from the cost of the uncertainty.

arcs.append((src,dest,int(cap/(N*QUANTIZATION)),mu*(i+1)*unit_cost
+ fee_rate_ppm))

2. the cost from the routing fees is the same on each segment of the
piecewise linearization which makes a lot of sense because the current
feerate is indeed a unit cost that does not change with how heavily you
plan to use a channel independently how the cost that comes from the
probability grows.

With respect to other features I guess the entire topic of feature
engineering for our cost function should be a separate threat / topic and
line of research but as you asked I will give you some short thoughts on
this here:

As pointed out to the c-lightning team in
https://github.com/ElementsProject/lightning/pull/4771#issuecomment-930173831
and
the following comment I believe that optimizing for CLTV is a poor choice
and in min cost flow computations it might very well be non linear anyway
and thus tricky to include in a meaningful way.Sure one could transform it
to a unit cost by doing something like CLTV*max_cap/cap and add it to the
cost function like the ppm. But I still do not really see why this is
useful. On the other hand I very much believe we should start to
investigate features that predict channel latency to setup / settle an HTLC
as suggested in the last paragraph of this comment
https://github.com/lightningdevkit/rust-lightning/issues/1170#issuecomment-972396747
However
I fear that latency measures again are non linear though they could again
be translated to a unit cost with the same trick as the CLTV.

> 6) Non-Zero Base Fee
>
> See Twitter [4].
>
> According to Stefan [5] it should be possible to integrate ZmnSCPxj's ideas
> to make this work with non-zero base fees. How?
> Simpler approach: Twitter [6].
>

I don't have much to add to the base fee discussion at this point besides
the emphasize that the above described linearization trick for CLTV or
latency based features will not properly work for the base fee because one
actually has to pay the full base fee at the end - independently of how
much you saturate the channel. This might mess up the optimization.

> 7) Private Channels
>
> [very niche topic, not really that interesting nor urgent]
>
> I'm a fan of adding private channels to provide more outbound liquidity,
> mainly to reduce gossip and hide my intentions. If my total liquidity to
> some peer is below the amount announced in public channels, I don't see
> any meaningful complication. However, I might have a public channel of
> size N and several private channels bringing my local liquidity to some
> value >N. It's rather obvious that not announcing this fact is a bad
> idea, as any #pickhardtpayments implementation would think I have 0-N on
> my side of the channel(s). Assuming I'm willing to accept this tradeoff,
> do you see other complications or issues with hidden liquidity?
>
> My gut feeling is that this isn't an issue, at all, as channel balances
> change all the time, which is something the algorithm already has to
> deal with.
>

As you noted from a probabilistic payment delivery point of view I might be
interested in signaling all the liquidity that is being provided between
two peers and I shoot myself if I hide it. That being said you might have
reasons to do so and additionally I never rejected the idea to extend the
current probabilistic model with a probabilistic node or channel provenance
value that would be similar to the ideas in lnd's mission control or the
routescore by https://lnrouter.app. Given the fact that with hidden
liquidity the probabilities are constantly underestimated such a provenance
score will probably be higher than the one of other channels fixing the
"introduced issue" of hidden liquidity

8) Quality of Approximation
>
> There are some problems in computer science that are hard/impossible to
> approximate, in the sense that any kind of deviation from the optimum
> could cause the computed results to be extremely bad. Do you have some
> idea (or proof) that your kind of approximation isn't causing a major
> issue? I guess a piece-wise linearization with an infinite number of
> pieces corresponds to the optimal result. Given a finite number of
> pieces, how large is the difference to the optimum?
>

We don't have to go to infinite to find a solution without error. We can
stay with the finite number that corresponds to the channels capacity and
make segments of 1 satoshi each encoding what this satoshi would actually
cost in the original function (assuming all values in the original function
were integers).

I have not spent the time to properly express the error in dependence of
the number of segments N or even empirically study its tradeoffs in
practice (as I haven't even included the optimal piecewise linearization
yet) However the actual flow (see below) as well as the results from Dr.
Martin Berger
https://lists.linuxfoundation.org/pipermail/lightning-dev/2022-March/003513.html
indicate
that the error should be possible to be managed and stay small enough in
practice. That being said and as above it certainly makes sense to invest a
bit of time on how to conduct the piecewise linearization properly.

Output of the computed flow from the iPython notebok with a piece wise
linearization of 5 equal sized segments per channel

Planning to deliver 0.50 BTC from 5051(03efccf...) to 13006
(021c97a...) via an approximated optimally reliable payment flow...

Runtime of flow computation: 0.85 sec
Minimum approximated quadratic cost: 815932

Arc Flow / Capacity probability Fee (sats)
5051 -> 8463 6700000 / 16777215 0.600649 8957.900000
5051 -> 3437 1800000 / 9000000 0.800000 2406.600000
5051 -> 9162 1240000 / 6200000 0.800000 1657.880000
5051 -> 14746 6700000 / 16777215 0.600649 8957.900000
5051 -> 14832 2000000 / 10000000 0.800000 2674.000000
6257 -> 14832 3350000 / 2411344242 0.998611 335.100000
14832 -> 12446 5350000 / 6200000000 0.999137 6.350000
7870 -> 6257 3350000 / 20000000 0.832500 34.500000
14746 -> 12446 6700000 / 100000000 0.933000 3350.000000
7914 -> 13006 6700000 / 200000000 0.966500 33501.000000
5051 -> 550 3300000 / 15000000 0.780000 4412.100000
11396 -> 13192 6700000 / 1000000000 0.993300 2010.000000
5051 -> 11396 6700000 / 16777215 0.600649 8957.900000
7199 -> 9162 3350000 / 445000000 0.992472 848.550000
9162 -> 12446 16590000 / 3900000000 0.995746 17.590000
5051 -> 13287 2000000 / 10000000 0.800000 2674.000000
5051 -> 1843 6700000 / 16777215 0.600649 8957.900000
5051 -> 11257 3350000 / 16777215 0.800324 4478.950000
1953 -> 12446 2160000 / 750000000 0.997120 1080.000000
550 -> 1843 3300000 / 30000000 0.890000 1155.000000
12756 -> 12446 2000000 / 16775679 0.880780 301.000000
5051 -> 12756 2000000 / 10000000 0.800000 2674.000000
3437 -> 12446 1800000 / 400000000 0.995500 225.001000
6713 -> 7914 6700000 / 200000000 0.966500 837.501000
11257 -> 7199 3350000 / 16777215 0.800324 663.300000
10914 -> 9162 2000000 / 500000000 0.996000 1000.001000
5051 -> 7870 3350000 / 16777215 0.800324 3.781000
8463 -> 5288 6700000 / 500000000 0.986600 837.501000
2781 -> 10914 2000000 / 500000000 0.996000 401.000000
13192 -> 6713 6700000 / 500000000 0.986600 16750.000000
32 -> 2781 2000000 / 210000000 0.990476 200.000000
5051 -> 9530 2160000 / 10811137 0.800206 2.591000
9530 -> 1953 2160000 / 16777215 0.871254 438.336000
1843 -> 9162 10000000 / 300000000 0.966667 2500.000000
5051 -> 14642 2000000 / 10000000 0.800000 2.431000
14642 -> 13006 2000000 / 50000000 0.960000 6001.000000
13287 -> 32 2000000 / 16777215 0.880791 500.000000
12446 -> 13006 34600000 / 200000000 0.827000 103766.400000
5288 -> 13006 6700000 / 100000000 0.933000 15216.700000

Probability of entire flow: 0.0032
Total fee: 248793.763 sats
Effective fee rate: 0.498 %
Arcs included in payment flow: 39

Don't get confused by a low probability. The first attampt always has
high uncertainty. We will learn fast in each consequitive round.

With kind regards Rene Pickhardt

--
> Dr. Carsten Otto
> carsten at c-otto.de
> https://c-otto.de
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

--
https://www.rene-pickhardt.de
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220314/77da14bd/attachment-0001.html>;

📅 Original date posted:2022-03-11 📝 Original message: Dear ...

2023-06-09T13:05:29Z

In reply to nevent1q…ccxz
_________________________

📅 Original date posted:2022-03-11
📝 Original message:
Dear fellow Lightning Developers,

I am pleased (and a bit proud) to be able to inform you that I finally
found a quick way to approximate the slow minimum convex cost flow
computation. This is necessary for optimally reliable and cheap payment
flows [0] to deliver large multi part payments over the Lightning Network.
The proposed solution happens via piecewise linearization [1] of the min
cost flow problem on the uncertainty network which we face in order to
compute the optimal split and planning of large amount multi part payments.
The notion of "optimal" is obviously subjective with respect to the chosen
cost function. As known we suggest to include the negative logarithm of
success probabilities based on the likelihood that enough liquidity is
available on a channel as a dominant feature of the used cost function. We
give the background for this in [2] which since then has already been
picked up by c-lightning and LDK. The c-lightning team even published
benchmarks showing significant improvement in payment speed over their
previously used cost function [2b].

Let me recall that one of the largest criticisms and concerns of our
approach to use minimum cost flows for payment delivery back in July /
August last year (especially by the folks from lightning labs) was that the
min cost flow approach would be impractical due to run time constrains.
Thus I am delighted that with the now published code [3] (which has exactly
100 lines including data import and parsing and ignoring comments) we are
able to compute a reasonable looking approximation to the optimal solution
in a sub second run time on the complete public channel graph of the
Lightning Network. This is achieved via piecewise linearization of the
convex cost function and invoking of a standard linear min cost flow solver
[4] for the linearized problem. This works quickly despite the fact that
the piecewise linearization adds a significant higher amount of arcs to the
network and blows up the size of the network on which we solve the min cost
flow problem. This makes me fairly certain that with proper pruning of the
graph we might even reach the 100 millisecond runtime frontier, which would
be far faster than what I dreamed & hoped to be possible.

The currently widely deployed Dijkstra search to generate a single
candidate path takes roughly 100ms of runtime. It seems that with the
runtime of the piecewise linearized problem the min cost flow approach is
now competitive from a runtime perspective. The flow computation is still a
bit slower than Dijkstra in both theory and practice. However the piecewise
linearized min cost flow has the huge advantage that it generates several
candidate paths for a solid approximation of the optimal MPP split.
Remember the exact min cost flow corresponds to the optimal MPP split. The
later was not used so far as the min cost flow was considered to be too
slow. Yet the question how to split seems to be addressed as issues in
implementations [5][6][7] and acknowledged to be complicated (especially
with respect to fees) in [8]. This result is btw of particular interest for
LSPs. If an LSP has to schedule x payments per second it can just do one
flow computation with several sink nodes and plan all of those payments
with a single min cost flow computation. This globally optimizes the
potentially heavy load that LSPs might have even if all payments were so
small that no splitting was necessary.

The iPython notebook which I shared contains about 1 page to explain how to
conduct a piecewise linear approximation. The quality of the approximation
is not the major goal here as I am just focused to demonstrate the run time
of the approach and the principle how to achieve this runtime. Thus I do
the piecewise linear approximation very roughly in the published code.
Selecting the optimal piecewise approximation [9] will not change the the
runtime of flow computation but only blows up the code to prepare the
solver. This is why I decided to keep the code as simple and short as
possible even if that means that the approximation will be not as close to
the optimum as it could be in practice. For the same reason I did not
include any code to update the uncertainty network from previously failed
or successful attempts by using conditional success probabilities P(X>a |
min_liquidity < X < max_liquidity ). People who are interested might look
at my hands on instructions and explanations for coders in this
rust-lightning issue [10]. The folks from LDK have picked this up and
implemented this already for single path payments in [11] which might be
relevant for people who prefer code over math papers. An obvious
optimization of the piece wise linearization would be to chose the first
segment of the piecewise linear approximation with a capacity of the
certain liquidity and a unit cost of 0 for that piece.

Our original papers describe everything only from a theoretical point of
view and with simulations. However our mainnet experiments from July last
year [12] indicated that we easily have been able to deliver for example
0.3679 BTC via a couple rounds of min cost flow computations through the
Lightning Network (given we accept the computational waiting time). This
experiment was too slow to be used in practice but confirmed our model and
approach to deliver such large amounts. Given that the min cost flow
computation is now significantly below the median onion round trip times it
is my understanding that with these newly introduced techniques we should
be able to deliver substantial monetary amounts over the Lightning Network
within a couple of seconds despite the uncertainty about the Liquidity in
remote channels. While I am very excited about the result I think some
principle roadblocks of the protocol are becoming more and more visible:

1. The well known issue of hanging HTLCs. As of now I believe onion
messages would be great to acknowledge incoming HTLCs of multipart payments
but I certainly believe this will not be sufficient if we don't go for a
cancelable & stuckless payment protocol as suggested in [13].
2. I believe the Lightning Network Protocol should be able to handle
redundant overpayments as suggested in [14] and [15] (The later paper
derives independently of us the same success probabilities and tries to
maximize them!). This would allow to transform the problem of finding a
flow that maximizes the success probability to a flow that expects to
deliver the amount of the invoice (when redundancy is applied) with a
single round of min cost flow computation on average.

Last but not least, please allow me to make a short remark on the (still to
me very surprisingly controversial) base fee discussion: For simplicity I
did not include any fee considerations to the published code (besides a fee
report on how expensive the computed flow is). However in practice we wish
to optimize at least for high reliability (via neg log success
probabilities) and cheap fees which in particular with the ppm is very
easily possible to be included to the piece wise linearized cost function.
While for small base fees it seems possible to encode the base fee into the
first segment of the piecewise linearized approximation I think the base
fee will still be tricky to be handled in practice (even with this
approximation). For example if the base fee is too high the "base fee
adjusted" unit cost of the first segment of the piecewise linearized
problem might be higher than the unit cost of the second segment which
effectively would break the convexity. Thus I reiterate my earlier point
that from the perspective of the year long pursued goal of optimizing for
fees (which all Dijkstra based single path implementations do) it seems to
be best if the non linearity that is introduced by the base fee would be
removed at all. According to discussions with people who crate Lightning
Network explorer (and according to my last check of gossip) about 90% of
channels have a base fee of 1 sat or lower and ~38% of all channels already
set their base fee away from the default value to 0 [16].

I hope this mail was useful for you. Feel free to ask me anything if there
should be questions or if you need help to integrate those results into
your software!

With kind regards Rene Pickhardt

[0]: https://arxiv.org/abs/2107.05322
[1]: https://en.wikipedia.org/wiki/Piecewise_linear_function
[2]: https://arxiv.org/abs/2103.08576
[2b]:
https://medium.com/blockstream/c-lightning-v0-10-2-bitcoin-dust-consensus-rule-33e777d58657

[3]:
https://github.com/renepickhardt/mpp-splitter/blob/master/Minimal%20Linearized%20min%20cost%20flow%20example%20for%20MPP.ipynb
[4]: https://developers.google.com/optimization/flow/mincostflow
[5]: https://github.com/lightningnetwork/lnd/issues/4203
[6]: https://github.com/lightningdevkit/rust-lightning/issues/1276
[7]: https://github.com/ElementsProject/lightning/issues/4753
[8]: https://github.com/ACINQ/eclair/pull/1427
[9]: http://www.iaeng.org/publication/WCECS2008/WCECS2008_pp1191-1194.pdf
[10]:
https://github.com/lightningdevkit/rust-lightning/issues/1170#issuecomment-972396747
[11]: https://github.com/lightningdevkit/rust-lightning/pull/1227
[12]: https://twitter.com/renepickhardt/status/1418849788531990530
[13]:
https://lists.linuxfoundation.org/pipermail/lightning-dev/2019-June/002029.html
[14]: https://berkeley-defi.github.io/assets/material/1910.01834.pdf
[15]: https://dl.acm.org/doi/10.1145/3479722.3480997
[16]: https://lnrouter.app/graph/zero-base-fee

--
https://ln.rene-pickhardt.de
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220311/0ba5d2d0/attachment-0001.html>;

📅 Original date posted:2021-11-15 📝 Original message: Dear ...

2023-06-09T13:04:24Z

In reply to nevent1q…hhzf
_________________________

📅 Original date posted:2021-11-15
📝 Original message:
Dear Joost,

First I am happy that you also agree that reliability can and should be
expressed as a probability as discussed in [0].

The problem that you address is that of feature engineering[1]. Which
consists of two (or even more) steps:

1.) Feature selection: That means in payment delivery we will compute a min
cost flow [2] with a chosen cost function (historically people used
dijkstra seach for single paths with the cost function representing the
weights on the edges of the graph -which is what most folks currently still
do). While [2] and I personally agree with you that the cost function
should be a combination the two features fees and reliability (as in
successprobability) Matt Corallo righfully pointed out [3] that other
features might be chosen in the future to deliver more optimal results. For
example implementations currently often use CLTV as a feature (which I
honestly find horrible) and I am currently investigating if one could add
latency of channels or - for known IP addresses - either the geo distance
or IP distance.

2.) Combining features: This is the question that you are asking. Often
people use a linear weighted sum to combine features. This is what often
happens implicitly in neural networks. While this is often good enough and
while it is often practical to either learn the weights or give users a
choice there are many situation where the weighted linear sum does not work
well with the selected features. An example for the weighted sum is the
risk-factor in c-lightning that could have been used to decide if one
wanted the dijkstra seach to either optimize for CLTV delta or for paid
routing fees. Also in our paper [2] in which we discuss the same two
features that you mentioned we explain how a linear sum of two features can
be optimal due to the lagrangian bounding principle. However in practice
(of machine learning) it has been shown that using the harmonic mean [4]
between features often works very well without the necessity to learn a
weight / parameter. This has for example been done when c-lightnign
recently switched to probabilistic path finding [5]. In this thread you
find a long discussion and evaluation how the harmonic mean outperformed
the linear sum.

I think the main issue that you address here is that there is no universal
truth for situations like this. In practice only tests and experience will
help us to make good decisions.

with kind Regards Rene

[0]:
https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-March/002984.html
[1]: https://en.wikipedia.org/wiki/Feature_engineering
[2]: https://arxiv.org/abs/2107.05322
[3]:
https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-September/003219.html
[4]: https://en.wikipedia.org/wiki/Harmonic_mean
[5]: https://github.com/ElementsProject/lightning/pull/4771

On Mon, Nov 15, 2021 at 4:26 PM Joost Jager <joost.jager at gmail.com> wrote:

> In Lightning pathfinding the two main variables to optimize for are
> routing fee and reliability. Routing fee is concrete. It is the sat amount
> that is paid when a payment succeeds. Reliability is a property of a route
> that can be expressed as a probability. The probability that a route will
> be successful.
>
> During pathfinding, route options are compared against each other. So for
> example:
>
> Route A: fee 10 sat, success probability 50%
> Route B: fee 20 sat, success probability 80%
>
> Which one is the better route? That depends on user preference. A patient
> user will probably go for route A in the hope of saving on fees whereas for
> a time-sensitive payment route B looks better.
>
> It would be great to offer this trade-off to the user in a simple way.
> Preferably a single [0, 1] value that controls the selection process. At 0,
> the route is only optimized for fees and probabilities are ignored
> completely. At 1, the route is only optimized for reliability and fees are
> ignored completely.
>
> But how to choose between the routes A and B for a value somewhere in
> between 0 and 1? For example 0.5 - perfect balance between reliability and
> fee. But what does that mean exactly?
>
> Anyone got an idea on how to approach this best? I am looking for a simple
> formula to decide between routes, preferably with a reasonably sound
> probability-theoretical basis (whatever that means).
>
> Joost
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

--
https://www.rene-pickhardt.de
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20211115/392fcbf0/attachment-0001.html>;

📅 Original date posted:2021-08-26 📝 Original message: Dear ...

2023-06-09T13:03:35Z

In reply to nevent1q…ss9g
_________________________

📅 Original date posted:2021-08-26
📝 Original message:
Dear fellow lightning developers,

with a mixture of shock and disbelief I have been following the (semi)
public discussions for the last 6 weeks and the reaction of some companies
/ people that reached out to me. I have to say I am really surprised by the
amount of hesitation that - despite obvious and overwhelming mathematical
evidence - a small group of people demonstrated in response to our
results. While I cannot make any sense of this I decided to post here
despite the fact that I believe everything that needed to be said is
already written and explained clearly in our paper [0] about optimally
reliable and cheap payment flows on the Lightning Network. My hope is that
this mail can help us to

* clarify a few misunderstandings
* stop having opinionated discussions about mathematical facts
* find a quick agreement on how to move on

As far as I can tell currently all implementations use some form of
Dijkstra algorithm in payment delivery with a strong emphasize on finding
paths with cheap fees. The reason seems to be that it is a well established
assumption that users would prefer a cheap solution on the market of
offered routing fees. (while routing nodes of course try to offer fees that
maximize their earnings)

If we look at the mentioned paper there are several results (I will soon
publish a separate mail here discussing some of the more interesting
results and consequences but I decided to split it and put this one here
first as it seemed to be more urgent). One of the results which I will
discuss now is the realization that - given the current fee function -
finding the cheapest payment flow is an NP-hard problem because the fee
function is neither linear nor convex.

Our fee function is `f(x) = rx + b` where `r`= fee rate, `b`=base fee and
`x` is the amount we want to send.

As we thought it was obvious that the function is not linear we only
explained in the paper how the jump from f(0)=0 to f(1) = ppm+base_fee
breaks convexity. But as the question came up several times (for example
here [1]) I want to stress that the fee function - despite looking like a
straight line - is **not linear**. While writing this post I realized that
the issue might be that the concept of linearity [2] from the field of
linear algebra seems to be intermixed in the english language and American
school system with the concept of linear polynomials / linear functions
[3]. So maybe that is part of the problem that emerged in previous
discussions.

When I write linear here I am referring back to the concept of linearity
from linear algebra (c.f.: [2]) which btw seems to be also the main reason
why schnorr signatures [2b] are so powerful that everyone is happy they
will find their way into bitcoin. We know that a necessary condition for a
function to be called linear is that the following property holds:

f(x+y)=f(x)+f(y)

however if we look at our setting we see that:

f(x+y) = r(x+y) + b

and

f(x)+f(y) = rx+b + ry+b = r(x+y) + 2b

equality (and thus linearity) only holds if

r(x+y)+b = r(x+y) + 2b <==> b = 2b

the later is true if and only if `b=0`. In other words: **Our current fee
function is only linear if we set the base fee to zero.**

On the other side we refer to research in our paper that shows that an
optimal solution (in this case optimal means cheapest) cannot be found if
the fee function is neither linear nor convex. I think one of the biggest
misunderstandings that I saw in the discussions is that people seem to have
thought this has something to do with our new / proposed method. But as far
as I can tell it does absolutely not have any connection to it. Instead and
as most of you probably know the property to be an NP-hard problem is
completely independent of the used algorithm. This is why in the paper we
suggested to drop the base_fee and mentioned in the German Podcast that the
same effect could already be achieved by node operators today by setting
their base fee to 0. it is also the reason why we asked before we published
the paper why the base fee was introduced and what purpose it served [4].

I am so surprised by some of the resistance because some of the biggest
talking points by Lightning Network critics in the past have been that
routing is either not solved or if it was solved it would be NP-hard. In
our paper we show that delivering a payment can always be modeled as a min
cost flow and the resulting optimization problem will indeed be NP-hard
depending on the cost function. Given the current fee function with a non
zero base fee and the goal of minimizing fees that is followed by all
implementations I have to agree with such critics and say yes, in that
particular case delivering a payment optimally is an NP-hard problem.

Luckily there are things we can do about it:

1.) we can decide to have a different optimization goal. For example in the
Paper we used a previously introduced probabilisitic model that optimizes
for reliability instead of fees and we show that in that case the function
is convex and a polynomial exact algorithm is known and exists.
2.) If we want to optimize for fees which - given current implementations
for single payments and the overwhelming feedback of users - seems to be
the case we can modify the fee function to be either convex or linear. As
everyone who read until here knows the later can be achieved by just
removing the base fee.
3.) Of course we can follow the ideas of Matt Corallo and Olaoluwa
Osuntokun to do more research. I mean there is a nice 1 million dollar
bounty [5] for the person who finds a polynomial solution to NP-hard
problems or proofs that such solutions cannot be found in polynomial time
(which if you ask me personally is way more probable to happen but I
usually try not to make wild conjectures about the future).
4.) We can try to find linear approximations so that we can have more
efficient algorithms to approximate such problems which I guess is what
Matt and Olaoluwa might have meant when they suggested to do more research.

As I was struggeling with the problem for over two years I personally have
a pretty strong opinion about the 4 potentials paths:

@1: We already describe in the paper that reliability should be included
when deciding which channels to use while fees should not be neglected.
Thus I tend to say: Yes let's modify the goal in a way that we still keep
optimizing for cheap delivery which by the end means keep a fee function
but NOT the current fee function.
@2: Seems like a total no-brainer to modify the fee function to drop the
base fee because I believe nobody wants to drop the cheap requirement for
payments and because of what I will say in reply to 3 & 4.
@3: If you want to find solutions to hard problems I suggest to find a
solution to the discrete logarithm instead of studying P=NP. The bounty is
the the security of bitcoin and thus much higher than just the million
dollar that you might get for the the solution to the P=NP problem. Ok,
jokes aside of course doing research is always a good idea and we laid out
a clear roadmap for further R&D in the paper that still holds even if we
all agree that zerobasefee is a good thing. I am still seeking funding from
the bitcoin / lightning network community to be able to continue research
and development in this field so I full appreciate the suggestion by Matt
and Olaoluwa who asked for more research and I hope the community will
entrust me to lead such research. I can understand that people hesitated to
support my work back in 2019 when I decided to focus on the reliability
question and it was a gamble if I could deliver but I sincerely hope with
my achievements that I lay out in full on https://ln.rene-pickhardt.de that
there will be more support in the future (not only for me).
@4: People in operation research have tried this for many years. In
practice problems are being linearized all the time. But in the best case
the models are already chosen to be linear and one hopes that such models
are close enough to reality. We have been given a gift that the uncertainty
in our channels naturally came as a convex cost function which makes the
reliability goal somewhat easy to compute. More importantly it is a huge
gift that it is just for us (users and developers) to decide how we want to
have the fee structure on the Lightning network. A linear solution could
just be agreed upon / recommended but of course we can keep a non-linear
version and struggle around with approximations to handle np-hardness. For
me it is so obvious what one would want to do that I will conclude with the
subject line of this mail as a rethorical question "Do we really want users
to solve an NP-hard problem when they wish to find a cheap way of paying
each other on the Lightning Network?"

In the recent Optech newsletter [6] Olaoluwa Osuntokun is said to have
"emphasized a point made earlier that there’s no clear current need for
node operators to change a parameter today for a new pathfinding algorithm
that nobody is currently ready to use in production". First of all I want
to emphasize that Stefan and I have used this in production and have
verified that our method works - as predicted in the paper - much better
than the pay implementation used in LND (I will write more about that in
the other mail that I already mentioned) But way more importantly (and as
explained in this mail and our paper) this parameter change is not about
some novel algorithm that we propose but rather about the mathematical
structure of the optimization problem that we want(?) users to face when
delivering payments (and that all implementations already presented to the
users in the single path payment case). So I very strongly disagree with
the statement that there is no clear need to change something. Actually I
think there is an exceptionally strong need to change something and very
good reason to do so rather sooner than later.

That being said the high reliability and large amounts that we could
deliver through the use of our method with a standard min convex cost flow
solver as the algorithm comes from the fact that we used probablistic path
finding [7] as the cost function. As demonstrated last march this already
works much better for single paths when used as a weight in dijsktra than
the stuff current implementations do. Also it did not create such a
discussion when I shared the results about probabilisitic pathfinding last
march and already mentioned that I only need to finish the work on optimal
splitting. So for the higher reliability and larger amounts alone that our
method can do, we do not even need a zerobasefee. This is because
probabilistic path finding merely models the uncertainty of the liquidity
in the channels to decide which channels to use with what amounts in
payment delivery to maximize the success probability. The only reason why
we mention (zerobase)fees is because of @1 and the assumption that node
operators would arbitrarily increase their fees on channels if we would not
take fees into consideration when deciding which channels to use. Since the
basefee kicks in and breaks the nice mathematical properties we addressed
it.

Btw I have a suggestion to node operators who seek channel partners and
nodes with whom to open channels. If you don't want your customers / users
be required to solve or approximate NP-hard problems you could open your
channels with other nodes who already support zero base fee on their
channels. As of now this seems to be a clear indicator that those nodes
care about reliability and try to provide a good & honest service to the
network. As more and more wallets should use optimal payment flows it is
also reasonable to expect that a lot of routing is likely to happen in the
#zerobasefee part of the network which is already pretty large [8]. At
least when I make a payment this part of the network is where I look first
to deliver the payment. Also connecting to nodes that indicate via
zerobasefee that they care seems at least to me personally way more
reasonable than following some random intransparent score that assigns
numbers to nodes.

As said initially I thought I already had said everything which is why -
unless substential new information comes up - I will most likely not engage
into further bike shedding discussions on the zero base fee discussions.
This is now for the community to decide and not for me to argue.

with kind regards Rene

[0]: https://arxiv.org/abs/2107.05322
[1]:
https://bitcoin.stackexchange.com/questions/108018/would-a-min-fee-setting-for-lightning-channels-make-sense#comment124039_108325
[2]: https://en.wikipedia.org/wiki/Linearity
[2b]:
https://courses.csail.mit.edu/6.857/2020/projects/4-Elbahrawy-Lovejoy-Ouyang-Perez.pdf
[3]: https://en.wikipedia.org/wiki/Linear_function
[4]:
https://bitcoin.stackexchange.com/questions/107311/why-was-the-base-fee-for-the-routing-fee-calculation-of-the-lightning-network-in
[5]:
https://en.wikipedia.org/wiki/P_versus_NP_problem#Results_about_difficulty_of_proof
[6]: https://bitcoinops.org/en/newsletters/2021/08/25/
[7]:
https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-March/002984.html
[8]: https://lnrouter.app/graph/zero-base-fee
--
https://www.rene-pickhardt.de
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20210826/7fde2220/attachment-0001.html>;

📅 Original date posted:2021-06-30 📝 Original message: Hey ...

2023-06-09T13:02:48Z

In reply to nevent1q…l3sc
_________________________

📅 Original date posted:2021-06-30
📝 Original message:
Hey everyone,

just for reference when I was new here (and did not understand the
processes well enough) I proposed a similar idea (called LIP) in 2018 c.f.:
https://lists.linuxfoundation.org/pipermail/lightning-dev/2018-July/001367.html

I wonder what exactly has changed in the reasoning by roasbeef which I will
repeat here:

*> We already have the equiv of improvement proposals: BOLTs. Historically*

>* new standardization documents are proposed initially as issues or PR's when *

>* ultimately accepted. Why do we need another repo? *

As far as I can tell there was always some form of (invisible?) barrier to
participate in the BOLTs but there are also new BOLTs being offered:
* BOLT 12: https://github.com/lightningnetwork/lightning-rfc/pull/798
* BOLT 14: https://github.com/lightningnetwork/lightning-rfc/pull/780
and topics to be included like:
* dual funding
* splicing
* the examples given by Ryan

I don't see how a new repo would reduce that barrier - Actually I think it
would even create more confusion as I for example would not know where
something belongs. That being said I think all the points that are
addressed in Ryan's mail could very well be formalized into BOLTs but maybe
we just need to rethink the current process of the BOLTs to make it more
accessible for new ideas to find their way into the BOLTs? One thing that I
can say from answering lightning-network questions on stackexchange is that
it would certainly help if the BOLTs where referenced on lightning.network
web page and in the whitepaper as the place to be if one wants to learn
about the Lightning Network

with kind regards Rene

On Wed, Jun 30, 2021 at 4:10 PM Ryan Gentry via Lightning-dev <
lightning-dev at lists.linuxfoundation.org> wrote:

> Hi all,
>
> The recent thread around zero-conf channels [1] provides an opportunity to
> discuss how the BOLT process handles features and best practices that arise
> in the wild vs. originating within the process itself. Zero-conf channels
> are one of many LN innovations on the app layer that have struggled to make
> their way into the spec. John Carvalho and Bitrefill launched Turbo
> channels in April 2019 [2], Breez posted their solution to the mailing list
> for feedback in August 2020 [3], and we know at least ACINQ and Muun
> (amongst others) have their own implementations. In an ideal world there
> would be a descriptive design document that the app layer implementers had
> collaborated on over the years that the spec group could then pick up and
> merge into the BOLTs now that the feature is deemed spec-worthy.
>
> Over the last couple of months, we have discussed the idea of adding a
> BIP-style process (bLIPs? SPARKs? [4]) on top of the BOLTs with various
> members of the community, and have received positive feedback from both app
> layer and protocol devs. This would not affect the existing BOLT process at
> all, but simply add a place for app layer best practices to be succinctly
> described and organized, especially those that require coordination. These
> features are being built outside of the BOLT process today anyways, so
> ideally a bLIP process would bring them into the fold instead of leaving
> them buried in old ML posts or not documented at all.
>
> Some potential bLIP ideas that people have mentioned include: each lnurl
> variant, on-the-fly channel opens, AMP, dynamic commitments, podcast
> payment metadata, p2p messaging formats, new pathfinding heuristics, remote
> node connection standards, etc.
>
> If the community is interested in moving forward, we've started a branch
> [5] describing such a process. It's based on BIP-0002, so not trying to
> reinvent any wheels. It would be great to have developers from various
> implementations and from the broader app layer ecosystem volunteer to be
> listed as editors (basically the same role as in the BIPs).
>
> Looking forward to hearing your thoughts!
>
> Best,
> Ryan
>
> [1]
> https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-June/003074.html
>
> [2]
> https://www.coindesk.com/bitrefills-thor-turbo-lets-you-get-started-with-bitcoins-lightning-faster
>
> [3]
> https://lists.linuxfoundation.org/pipermail/lightning-dev/2020-August/002780.html
>
> [4] bLIP = Bitcoin Lightning Improvement Proposal and SPARK =
> Standardization of Protocols at the Request of the Kommunity (h/t fiatjaf)
>
> [5]
> https://github.com/ryanthegentry/lightning-rfc/blob/blip-0001/blips/blip-0001.mediawiki
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

--
https://www.rene-pickhardt.de
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20210630/eeeb1aa7/attachment-0001.html>;

📅 Original date posted:2020-07-17 📝 Original message: Hey ...

2023-06-09T13:00:35Z

In reply to nevent1q…zu33
_________________________

📅 Original date posted:2020-07-17
📝 Original message:
Hey Ankit,

The lightning network sees the possession of a preimage as a proof of
payment. And I believe everyone agrees that a court should rule in favor of
A forcing E to deliver the good or reimburse A. The reason is that
possession of the preimage matching the signed payment hash from E is a
much stronger evidence of A actually having paid than E claiming to not
have received anything.
This is also due to the fact that guessing the preimage can practically be
considered impossible (though there is a tiny likelihood)

If E breaches the protocol by giving the preimage to C (for free) instead
of claiming the money from D (and thus settling the Htlc) it will be
considered E's problem, that E did not get reimbursed but just gave out the
preimage for free. (actually E's so called "partner in crime" did get
reimbursed). Even if D would testify that E never settled the Htlc one
would wonder why E never settled the incoming htlc as they should only have
created a payment hash for which they know the preimage. Since A can
actually provide one it is again unlikely if E for example claims they just
used a random hash for which they didn't know the preimage because they
wanted to just see if A has enough liquidity.

With kind regards Rene

Ankit Gangwal <A.Gangwal at tudelft.nl> schrieb am Fr., 17. Juli 2020, 08:43:

> Consider A wants to send some funds to E.
>
>
>
> They don’t have a direct payment channel among them. So, they use a
> following path A-B-C-D-E. A is the sender of payment and E is final
> recipient.
>
>
>
> E sends the hash of a secret r to A, A passes on the hash to B, B to C, C
> to D, and D to E.
>
>
>
> E discloses the secret to C (a partner in crime with E) and E do not
> respond to D. C gives the secret to B (settling the HTLC between them).
> Then, B gives the secret to A (settling the HTLC between them).
>
>
>
> A sent (and lost) the money, as E denies receiving the money (and the
> promised service/good).
>
>
>
> How the lightening network sees this? Out of their control?
>
>
>
> --
>
> A_G
>
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20200717/e0c86579/attachment.html>;

📅 Original date posted:2020-06-17 📝 Original message: Hey ...

2023-06-09T13:00:23Z

In reply to nevent1q…q5u5
_________________________

📅 Original date posted:2020-06-17
📝 Original message:
Hey everyone and of course good morning ZmnSCPxj (:

about 11 months ago I discovered a potential blackmail attack with HTLCs
after answering this question on stack exchange (c.f
https://bitcoin.stackexchange.com/questions/89232/why-is-my-spendable-msat-much-lower-than-msatoshi-to-us/89235#89235).
This attack is similar to the one that was possible with tx malleability on
the funding transaction without the segwit upgrade (c.f.
https://commons.wikimedia.org/w/index.php?title=File:Introduction_to_the_Lightning_Network_Protocol_and_the_Basics_of_Lightning_Technology_(BOLT_aka_Lightning-rfc).pdf&page=126).
Meaning an attacker can force a victim to lose money and use this fact to
blackmail the victim, to potentially gain / steal some of the lost funds.

TL;DR:
=====
* Depending on the circumstances this attack allows an attacker to make
channel partners lose a substantial amount of BTC without substantial costs
for the attacker.
* Depending on the exact circumstances this could be for example ~0.15 BTC.
In particular it demonstrates why opening a channel is not an entirely
trustless activity.
* The attacker will reliably only be able to force the victim to lose this
amount of Bitcoin.
* It is not clear how in practice the attacker could gain this amount or
parts of it as this would involve not only game theory but also rather
quick communication between attacker and victim and customized Lightning
nodes which at least for the victim would be unlikely to exist.
* None of the suggested fixes seems to be satisfying though the current
solution of lowering the maximum amount of HTLCs that can concurrently be
in flight seems to be a reasonable start for now.

Timeline on Disclosure
=================
I have disclosed this attack on Sunday July 21st 2019 to Fabrice Drouin
(and shortly after to Christian Decker) in a phone call who in turn has
discussed it with people from the other implementations. From his feedback
I understood that people working on implementations have been more or less
aware of the possibility of this attack. Fabrice also mentioned that he
believed implementations currently try to mitigate this by setting low
limits of allowed / accepted HTLCs in flight. However at that time this was
only true for e-clair. It is now also true for c-lightning and as far as I
know still not true for lnd. Fabrice said that the people he talked to have
suggested that I should eventually describe the attack in public to raise
awareness (also from the group of node operators) for the problems related
to this attack. He also suggested that - if I wanted to - I should update
the rfc with recommendations and warnings. While I already have in mind
how to change the rfc I wanted to start the discussion first. Maybe some
people find better fixes than just a warning that I have in mind. So far I
didn't do anything because I wanted to also give lnd the chance to handle
the problem.

There are two reasons I disclose this attack today:
1.) I think almost 1 year is enough time to do something about it. The only
implementation that afaik didn't yet is lnd (see below) but I got roasbeefs
ok last week to go ahead and publish the attack anyway so that we can have
a broader discussion on mitigation strategies.
2.) The attack seems actually very similar to the one described in the
"Flood & Loot: A Systemic Attack On The Lightning Network" - paper which
came out 2 days ago (c.f.: https://arxiv.org/abs/2006.08513 ). I believe
any person reading that paper will understand the possibility of the attack
that I describe anyway so I believe it is now more or less public anyway
and thus time for an open / public discussion.

The main difference between the two attacks (if I understand this novel
paper correctly) is: In the "flood and loot"-attack one tries to steal the
HTLC output of the victims. Where in the "flood and blackmail"-attack that
I describe I try to to force the victim to lose almost all its funds due to
high on chain fees (Which I could use to blackmail the victim)

Description of the attack
===================
Let us assume the victim has funded a channel with an attacker meaning it
will have to pay the fees for the commitment transaction in case of a force
close.

During a fee spike (let us assume fee estimators suggest 150 sat / byte)
the attacker spams this channel with the maximum possible amount of HTLCs
that the protocol allows. The HTLCs can be of a small value but need to be
bigger than the dust limit so that additional outputs are actually added to
the commitment transaction which makes it quite large in Bytes. According
to the BOLTs these are 483 additional outputs to the commitment
transaction.
The direction of HTLCs are chosen so that the amount is taken from the
`to_remote` output of the attacker (obviously on the victims side it will
be the `to_local` output) For the actual attack it does not matter in which
direction the HTLCs are spammed but economically the direction I propose
makes even more sense for the attacker and can be achieved with circular
onions.

The attacked channel partner will happily - according to the protocol - use
a higher fee than the current fee rate. I quote from BOLT 02 which suggests
a buffer of a factor of 5
https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md#updating-fees-update_fee:

> The node responsible for paying the Bitcoin fee [...] SHOULD send
update_fee to ensure the current fee rate is sufficient (by a significant
margin) for timely processing of the commitment transaction. [...] Given
the variance in fees, and the fact that the transaction may be spent in the
future, it's a good idea for the fee payer to keep a good margin (say 5x
the expected fee requirement); but, due to differing methods of fee
estimation, an exact value is not specified.

This overpayment of fees will result in 750 sat / byte for this fee spike
scenario. This is by the way not completely unrealistic [I recently opened
a channel with 2.56 sat / byte (c.f.:
https://www.smartbit.com.au/tx/c0ac6cfe15e0d0c921362ab9fad998a8a8e16cd8d9d4159487dd69141ea2b9b0)
and the channel was force closed a couple minutes later due to an
implementation bug resulting in fees of 101.17 sat / byte (c.f.:
https://www.smartbit.com.au/tx/e32135315ec147bb27f771b2e15c7178ea573afd16cd4970bf814c9b18bc46e3
)

As far as I understand the appendix of BOLT 03 offered HTLCs are 43 Byte in
size (c.f.:
https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md#appendix-a-expected-weights)
resulting in the following additional fees for the spammed commitment
transaction:

fee (for 483 htlcs) = 483 * 43 byte * 750 sat / byte = 15576750 sat =
0.1557675 BTC

Additionally the victim will also have to swipe all offered HTLCs (which
will be additional costs but could be done once the fees came down) so we
neglect them.

Once all HTLCs are set up the attacker will stop signing commitment
transactions. In the beginning this is not suspicious as the HTLCs will
take some time to settle anyway. But also when fees go down and the victim
who funded the channel wants to update the fees the attacker will just not
be responsive. Eventually the victim will dare to force close the channel
with all those expensive HTLCs.

Knowing that this will happen and that the victim has to spend those funds
(publishing old state obviously does not work!) the attacker has a time
window to blackmail the victim outside of the lightning network protocol:
"Either you will pay those 0.1557675 BTC of fees or we will collaboratively
close the channel but that will cost you part of the amount you lost." Game
theory suggests that the attacker will be able to claim the major fraction
of the BTC (c.f. https://en.wikipedia.org/wiki/Ultimatum_game that are
frozen in tx fees as the victim effectively already has lost that money and
can only gain something back.

Thoughts
=======
You might say that the blackmail part of this attack is unrealistic as the
blackmailing person will not have enough time to successfully do the
blackmail as the channel that is not operational will fail.

1.) The only thing that lightning nodes might do is a fee update as the
htlcs stuffed the channel so I believe there is actually some time to
contact the victim.
2.) What if the attacker is a mining pool who is just interested in high
fees who does not even need to do the blackmailing stuff but will just
force close the channel once the htlcs are set up?
3.) The attacker might not even be interested in blackmailing the victim.
The attacker could just be interested in harming the victim. Though it
might certainly be a challenge to target a specific victim and trick it
into opening a channel with an attacker.

Also you might say that an attacker needs many incoming channels to execute
this attack. This can be achieved by gaming the autopilot. an attacker can
start by creating many channels making him a highly likely channel partner
for autopilot users (who will also fund the channel). Such a highly
connected node might also be interesting for non autopilot users.

Implementations
============

I looked at the code myself. I hope I do oversee things but to me it looks
like only eclaire was somehow mitigating this attack from being exploited.
(by a default config of 30 accepted htlcs which will protect the average
user and is much lower than the 483) and c-lightning has merged a patch
from me which I provided after I disclosed the attack:

## clightning:

c-lightning did not by default set a hard cap on htlcs before version 0.7.2
but then merged my patch
https://github.com/ElementsProject/lightning/pull/2858 which tried to
resemble the eclair defaults

## eclaire:

the max accepted htlc value per channel is set as a constant to 483 which
follows the recommendation of the BOLTs:
https://github.com/ACINQ/eclair/blob/e62adf2deae213d2cd0f2a6874227dcfc57880ae/eclair-core/src/main/scala/fr/acinq/eclair/channel/Channel.scala#L52

The value is tested against at:
https://github.com/ACINQ/eclair/blob/f724efaa76b256048de18f706e9cb58ecbebd6aa/eclair-core/src/main/scala/fr/acinq/eclair/channel/Helpers.scala#L99

and:
https://github.com/ACINQ/eclair/blob/f724efaa76b256048de18f706e9cb58ecbebd6aa/eclair-core/src/main/scala/fr/acinq/eclair/channel/Helpers.scala#L132

and:
https://github.com/ACINQ/eclair/blob/93d9369f900766171f2ddf579e8b12e28d8f0d25/eclair-core/src/main/scala/fr/acinq/eclair/channel/Commitments.scala#L154

However the default config file specifies a maximum amount of 30 accepted
htlcs at:
https://github.com/ACINQ/eclair/blob/9afb26e09c69dd5d6a14732baf5dcdf2b7a9142b/eclair-core/src/main/resources/reference.conf#L62

## lnd
It seems like lnd did not and last time I checked (maybe I oversaw
something) does not set a hard cap on htlcs by default. The way how I
understand the code they allow up to 483 htlcs by default:

The test when adding an htlc if it is beyond the maximum accepted values is
here:
https://github.com/lightningnetwork/lnd/blob/970d7604071baae227db42d4665ef9d1b56988e8/lnwallet/channel.go#L3795

the default configuration seems to be here (and in the most recent commit
the line still exists):
https://github.com/lightningnetwork/lnd/blob/8b04cfbf12f460853e8c55611cd1bba21b1510ef/input/size.go#L187

the software will accept up to 483 htlcs according to this line:
https://github.com/lightningnetwork/lnd/blob/111cbeaa990cba78563d6cc8c19b152e2d3042f6/lnwallet/reservation.go#L314

I could not find any spec lower than that in the suggested sample config
at:
https://github.com/lightningnetwork/lnd/blob/master/sample-lnd.conf

Ideas for Fixes
===========

I am completely unhappy with each of the following ideas. I hope you will
come up with smarter solutions. I believe the solution is not obvious. Thus
I thought it makes sense in the brainstorm phase to even post some ideas
with obvious drawbacks.

1. The current solution is to just not use up the max value of
htlc's. Eclaire and c-lightning by default only use up to 30 htlcs.
2. Probably the best fix (not sure if I understand the consequences
correctly) is coming from this PR to bitcoin core (c.f.
https://github.com/bitcoin/bitcoin/pull/15681 by @TheBlueMatt . If I get it
correctly with that we could always have low fees and ask the person who
want to claim their outputs to pay fees. This excludes overpayment and
could happen at a later stage when fees are not spiked. Still the victim
who offered the htlcs would have to spend those outputs at some time.
3. Don't overpay fees in commitment transactions. We can't foresee the
future anyway
4. Don't add htlcs for which the on chain fee is higher than the HTLCs
value (like we do with sub dust amounts and sub satoshi amounts. This would
at least make the attack expensive as the attacker would have to bind a lot
of liquidity.
5. Somehow be able to aggregate htlc's. In a world where we use payment
points instead of preimages we might be able to do so. It would be really
cool if separate HTLC's could be combined to 1 single output. I played
around a little bit but I have not come up with a scheme that is more
compact in all cases. Thus I just threw in the idea.
6. Split onchain fees differently (now the attacker would also lose fees by
conducting this attack) - No I don't want to start yet another fee
bikeshadding debate. (In particular I believe that a different split of
fees might make the Flood & Loot attack economically more viable which
relies on the same principle)

Independently I think we should have a hint in our readme file about where
and how people can disclose attacks and vulnerabilities. Implementations
have this but the BOLTs do not.

with kind regards Rene

--

https://www.rene-pickhardt.de

Skype: rene.pickhardt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20200617/07158fbc/attachment-0001.html>;

📅 Original date posted:2020-05-05 📝 Original message: Dear ...

2023-06-09T13:00:15Z

In reply to nevent1q…ttdl
_________________________

📅 Original date posted:2020-05-05
📝 Original message:
Dear Subhra,

as discussed bilaterally and after clarification of your question the
situation is as follows:

Let us assume A and B have a channel in which A has 4 tokens and B has 6
tokens

Now A offers an HTLC with the amount of 2 tokens and B accepts (receives)
the offer then A and B both have negotiated the HTLC output in the most
recent commitment transaction.

If A stops responding and B has to force close the channel a commitment
transaction with 3 UTXOs will hit the chain. One UTXO with 2 tokens
spendable by A, another one with 6 tokens spendable by B and the received
HTLC output with 2 tokens. This one can be spend by two different
conditions as in the offchain protocol

1.) Before the timelock of the HTLC has passed B can spend the output if B
knows his to_local HTLC secret AND the preimage. OR
2.) after the timelock A can spend the output if A knows the to_remote HTLC
secret.

the mechanism with HTLCs can be read upon in BOLT 2 (channel operation
https://github.com/lightningnetwork/lightning-rfc/blob/master/02-peer-protocol.md)
and the scripts can be seen in BOLT 3:
https://github.com/lightningnetwork/lightning-rfc/blob/master/03-transactions.md

A less technical summary that is more focused on explaining the concepts is
currently being developed in the routing chapter of mastering the lightning
network:
https://github.com/lnbook/lnbook/blob/43ce57298b4da345286ae3b53c42ea3eb9d9b056/routing.asciidoc

With kind regards Rene Pickhardt

On Tue, May 5, 2020 at 8:06 PM Subhra Mazumdar <
subhra.mazumdar1993 at gmail.com> wrote:

> Hi,
> I am having a doubt regarding force closure of channel. Suppose A->B
> there is an htlc which has been established for transfering fund. Now
> suppose for some unfortunate reason B doesnt have the witness to resolve
> htlc and the mean time A suffers crash fault. Then can B close the channel
> given that it has no way out of resolving the htlc due to lack of witness?
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20200505/ee92c436/attachment-0001.html>;

📅 Original date posted:2020-02-20 📝 Original message: Hey ...

2023-06-09T12:59:00Z

In reply to nevent1q…6cst
_________________________

📅 Original date posted:2020-02-20
📝 Original message:
Hey Rusty,

I was very delighted to read your proposal. But I don't see how you prevent
message spam. If I understand you correctly you suggest that I can
communicate to any node along a path of peer connections (not necessarily
backed by payment channels but kind of only known through channel
announcements of gossip) via onions and these onions which are send within
a new gossip message are not bound to any fees or payments.

Let's assume I just missed some spam prevention mechanism or that we can
fix them. Do I understand the impact of your suggestion correctly that I
could use this protocol to

1.) create a fee free rebalancing protocol? Because I could also attach a
new lightning message inside the onions that would allow nodes without
direct peer connection to set up a circular rebalancing path.
2.) have the ability to communicate with nodes further away than just my
peers - for example to exchange information for pathfinding and / or
autopilots?

With kind regards Rene

Rusty Russell <rusty at rustcorp.com.au> schrieb am Do., 20. Feb. 2020, 10:37:

> Hi all!
>
> It seems that messaging over lightning is A Thing, and I want to
> use it for the offers protocol anyway. So I've come up with the
> simplest proposal I can, and even implemented it.
>
> Importantly, it's unreliable. Our implementation doesn't
> remember across restarts, limits us to 1000 total remembered forwards
> with random drop, and the protocol doesn't (yet?) include a method for
> errors.
>
> This is much friendlier on nodes than using an HTLC (which
> requires 2 round trips, signature calculations and db commits), so is an
> obvious candidate for much more than just invoice requests.
>
> The WIP patch is small enough I've pasted it below, but it's
> also at https://github.com/lightningnetwork/lightning-rfc/pull/748
>
> diff --git a/01-messaging.md b/01-messaging.md
> index 40d1909..faa5b18 100644
> --- a/01-messaging.md
> +++ b/01-messaging.md
> @@ -56,7 +56,7 @@ The messages are grouped logically into five groups,
> ordered by the most signifi
> - Setup & Control (types `0`-`31`): messages related to connection
> setup, control, supported features, and error reporting (described below)
> - Channel (types `32`-`127`): messages used to setup and tear down
> micropayment channels (described in [BOLT #2](02-peer-protocol.md))
> - Commitment (types `128`-`255`): messages related to updating the
> current commitment transaction, which includes adding, revoking, and
> settling HTLCs as well as updating fees and exchanging signatures
> (described in [BOLT #2](02-peer-protocol.md))
> - - Routing (types `256`-`511`): messages containing node and channel
> announcements, as well as any active route exploration (described in [BOLT
> #7](07-routing-gossip.md))
> + - Routing (types `256`-`511`): messages containing node and channel
> announcements, as well as any active route exploration or forwarding
> (described in [BOLT #7](07-routing-gossip.md))
> - Custom (types `32768`-`65535`): experimental and application-specific
> messages
>
> The size of the message is required by the transport layer to fit into a
> 2-byte unsigned int; therefore, the maximum possible size is 65535 bytes.
> diff --git a/04-onion-routing.md b/04-onion-routing.md
> index 8d0f343..84eff9a 100644
> --- a/04-onion-routing.md
> +++ b/04-onion-routing.md
> @@ -51,6 +51,7 @@ A node:
> * [Legacy HopData Payload Format](#legacy-hop_data-payload-format)
> * [TLV Payload Format](#tlv_payload-format)
> * [Basic Multi-Part Payments](#basic-multi-part-payments)
> + * [Directed Messages](#directed-messages)
> * [Accepting and Forwarding a
> Payment](#accepting-and-forwarding-a-payment)
> * [Payload for the Last Node](#payload-for-the-last-node)
> * [Non-strict Forwarding](#non-strict-forwarding)
> @@ -62,6 +63,7 @@ A node:
> * [Returning Errors](#returning-errors)
> * [Failure Messages](#failure-messages)
> * [Receiving Failure Codes](#receiving-failure-codes)
> + * [Directed Message Replies](#directed-message-replies)
> * [Test Vector](#test-vector)
> * [Returning Errors](#returning-errors)
> * [References](#references)
> @@ -366,6 +368,13 @@ otherwise meets the amount criterion (eg. some other
> failure, or
> invoice timeout), however if it were to fulfill only some of them,
> intermediary nodes could simply claim the remaining ones.
>
> +### Directed Messages
> +
> +Directed messages have an onion with an alternate `hop_payload`
> +format. If this node is not the intended recipient, the payload is
> +simply a 33-byte pubkey indicating the next recipient. Otherwise, the
> +payload is the message for this node.
> +
> # Accepting and Forwarding a Payment
>
> Once a node has decoded the payload it either accepts the payment
> locally, or forwards it to the peer indicated as the next hop in the
> payload.
> @@ -1142,6 +1151,11 @@ The _origin node_:
> - MAY use the data specified in the various failure types for debugging
> purposes.
>
> +## Directed Message Replies
> +
> +Directed message replies are encoded the same way as failure messages,
> +except the contents is a directed message for the originator.
> +
> # Test Vector
>
> ## Returning Errors
> diff --git a/07-routing-gossip.md b/07-routing-gossip.md
> index ec1a8f0..4c2b836 100644
> --- a/07-routing-gossip.md
> +++ b/07-routing-gossip.md
> @@ -1,4 +1,4 @@
> -# BOLT #7: P2P Node and Channel Discovery
> +# BOLT #7: P2P Node and Channel Discovery and Directed Messages
>
> This specification describes simple node discovery, channel discovery,
> and channel update mechanisms that do not rely on a third-party to
> disseminate the information.
>
> @@ -31,6 +31,7 @@ multiple `node_announcement` messages, in order to
> update the node information.
> * [HTLC Fees](#htlc-fees)
> * [Pruning the Network View](#pruning-the-network-view)
> * [Recommendations for Routing](#recommendations-for-routing)
> + * [Directed Messages](#directed-messages)
> * [References](#references)
>
> ## Definition of `short_channel_id`
> @@ -1103,6 +1104,37 @@ A->D's `update_add_htlc` message would be:
> And D->C's `update_add_htlc` would again be the same as B->C's direct
> payment
> above.
>
> +# Directed Messages
> +
> +Directed messages allow peers to use existing connections to query for
> +invoices (see [BOLT 12](12-offer-encoding.md)). Like gossip messages,
> +they are not associated with a particular local channel. Like HTLCs,
> +they use [BOLT 4](04-onion-routing.md#directed-messages) protocol for
> +end-to-end encryption.
> +
> +Directed messages are unreliable: in particular, they are designed to
> +be cheap and not to need to be committed to a database (though an
> +implementation may choose to). Each one has an optional reply, which
> +is [onion encoded](04-onion-routing.md#directed-message-replies) 0just
> +like HTLC errors.
> +
> +## The `directed` and `directed_reply` Messages
> +
> +1. type: 385 (`directed`) (`option_directed_messages`)
> +2. data:
> + * [`1366*byte`:`onion_routing_packet`]
> +
> +1. type: 386 (`directed_reply`) (`option_directed_messages`)
> +2. data:
> + * [`sha256`:`onion_routing_packet_hash`]
> + * [`u16`:`len`]
> + * [`len*byte`:`reply`]
> +
> +## Requirements
> +
> +FIXME: similar to update_add_htlc and update_fail_htlc.
> +FIXME: define reasonable timeout after which you can forget if not
> replied?
> +
> ## References
>
> 1. <a id="reference-1">[RFC 1950 "ZLIB Compressed Data Format
> Specification version 3.3](https://www.ietf.org/rfc/rfc1950.txt)</a>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20200220/8274f3ba/attachment.html>;

📅 Original date posted:2020-02-09 📝 Original message: Good ...

2023-06-09T12:58:50Z

In reply to nevent1q…45f8
_________________________

📅 Original date posted:2020-02-09
📝 Original message:
Good Morning Cezary,

you might want to direct questions about understanding the lightning
network protocol like yours to https://bitcoin.stackexchanage.com as this
mailinglist is more devoted towards driving the development of the
protocol. Anyway here are the answers to your questions 2 and 3 and
probably also to the first one though I am not entirely sure if I
understand exactly what you are asking for. In case I misunderstood you
suggest to put follow up questions on stackexchange.

1. Is this possible that by sending funds without invoice, the last hub
> prepares the last HTLC with small amount to the payee? In other words - Can
> payee detect, that the last HTLC amount is smaller that it should be?
>

But in general the payee will only release the preimage for an invoice if
the payee is satisfied with the amount - which is usually specified in the
invoice. If you talk about keysend then the payee does not expect an amount
and will most likely release the preimage as the payee would consider this
to be free money

> 2. Are there additional data added to the end of onion encrypted list of
> HTLCs in order to prevent last hub to guess, that it is the last hub in the
> route?
>

yes the onions are always of constant size (20 * 65 Byte = 1300 Byte) This
process of padding is well described in the Sphinx paper
https://cypherpunks.ca/~iang/pubs/Sphinx_Oakland09.pdf and in BOLT 04
https://github.com/lightningnetwork/lightning-rfc/blob/master/04-onion-routing.md

3. When payment is during confirmation, are channels locked entirely, or
> only for the in flight payment amount? In other words - can single channel
> process more that single transaction at once?
>

HTLCs are additional outputs in the commitment transaction. The protocol
allows for up to 483 htlcs concurrently in flight as specified in BOLT 04 ("
max_accepted_htlcs is limited to 483 to ensure that, even if both sides
send the maximum number of HTLCs, the commitment_signed message will still
be under the maximum message size. It also ensures that a single penalty
transaction can spend the entire commitment transaction, as calculated in BOLT
#5
<https://github.com/lightningnetwork/lightning-rfc/blob/master/05-onchain.md#penalty-transaction-weight-calculation>;
.")

However the the standard of implementations and recommendation is 30. This
means that a single payment is not freezing the channel. It however "locks"
the amount of that payment which for the time until settlement cannot be
used by either party of the channel for other payments / activities.

with kind regards Rene

> I would like to kindly pleas to reply in simple words, as my English is
> still far from being perfect.
>
> Best Regards,
> Cezary Dziemian
>
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20200209/d8727a02/attachment-0001.html>;

📅 Original date posted:2019-07-15 📝 Original message: Dear ...

2023-06-09T12:55:34Z

In reply to nevent1q…rw2m
_________________________

📅 Original date posted:2019-07-15
📝 Original message:
Dear fellow BOLT devs,

in this mail I want to suggest a congestion and flow control mechanism to
improve the speed and reliability of multi path routing schemes. This is
the first of a couple of emails that I will write in the following weeks as
I have used my break in hospital not only to recover but to tinker quite
a bit about path finding and routing algorithms on the lightning network.

Problem statement
===============
Currently on the lightning network we have the issue of stuck payments [0].
As soon as an onion is sent it is out of the sender's control. This problem
seems to be in particular drastic if we wish to use Atomic Multi Path
routing [1] (which in the described form is not compatible with my
proposal. I believe my proposal should be compatible with the status quo of
base-AMP). The entire payments and HTLCs of a multipath payment will only
be settled once enough incoming HTLCs arrived at the recipient (meaning the
sum of amounts is bigger or equal to the amount specified in the invoice).
This has the following list of downsides:

- One malicious actor (who is just not forwarding the onion but also not
signaling an error) is enough to interrupt the entire payment process and
freeze all other HTLCs even of partial payments the actor is not part of.
- The entire payment process takes as long as `max_{p \in paths}(t(p))`
where `t(p)` is the time it took for path `p` to set up (and settle) HTLCs
- More HTLCs will be reserved by the network for a longer time. This means
more liquidity is bound / reserved and channels could even become unusable
if the 483 HTLC limit is reached.

Protocol Goals
===========
I looked at the windowing mechanism used in TCP to achieve congestion
control and transferred this concept to the setting of the Lightning
Network. This idea is motivated by the Spider Network paper [2] which
mentions that in a simulation the success rate of payments is increased
when changing the lightning network from a circuit switched payment process
(which we currently have with our atomicity requirements) to a packet
switched mechanism that includes congestion control (though in that
publication congestion control had a different semantics than in has in my
proposal).

Protocol Benefits
=============
- Improve the speed of multipath payments
- Reduce load from the network (in particular don't lock liquidity for such
a long time)
- less congestion at single nodes (I assume this is not a problem at this
point in time)
- more privacy (different preimages are used along different paths and
overall payments might become smaller or of uniform size)
- usual benefits from AMP

Protocol idea (base version)
=====================
Disclaimer: This base version has obvious drawbacks but I decided to
include it as it transports the idea.

A regular payment on the Lightning Network for amount `x` has a Payment
Hash `H` and a preimage `r`. If a recipient would now accept that this
payment could be split over up to `n` paths the recipient would create a
sha-chain of preimages and payment hashes with `n` elements

```
r_0 = rand()
H_0 = H(r_0)
r_{i+1} = H_i
H_{i+1} = H(r_{i+1})
```

The payment process is initiated by the recipient providing H_{n-1} and
signaling (in the invoice) that up to `n` preimages are available to
complete this payment.

A sender can now decide to split the payment amount `x` into `n` seperate
payments for example of the amounts `x/n` though different splits should be
possible. Once the preimage of the first partial payment is returned the
payer learns the payment hash wich can be used for the next partial
payment. (One issue is that while we have a proof of payment we do not
necessarily have a proof of amount - which is true for the regular
lightning case though with a single atomic payment this is not an issue as
the preimage will not be relased if the amount is too low. We could avoid
this issue by demanding that mulipath payments have to be at least of size
`x/n`)

This protocol makes the AMP process sequential and reduces the load from
the network. Congestion (which is a local problem of routing nodes) becomes
less likely if only HTLCs are locked up for a partial payment independent
of the success or failure of other partial payments. However in the base
version there is a severe downside:

**Sequential payments will make the payment process even longer since it is
not the max time needed over all payments but the sum of times needed.**

We can resolve this issue by introducing flow control via a windowing
mechanism and allowing concurrency of partial payments

Protocol Overview (suggested version)
==============================
Let us assume the receiving node supports a window size of `s` concurrent
payments. Now the payee will not only create one sha-chain of `n` payment
hashes as in the base version but `s` sha-chains of `n` payment hashes.
In the invoice we would now transport the following data:

* `n` (we need a different letter as n is already taken) = amount of
partial payments that are supported per payment hash
* `s` = number of concurrent payments supported (window size)
* `s` many `p` fields which contain the `s` different top elements H_{n-1}
for each sha-chain

Note that technically the payment amount could now be even split up into
`s*n` partial payments though I would recommend to still go with `n`
partial payments.

The advantage with this mechanism are the following:
* As long as less then `s` payments get stuck the protocol can continue to
deliver partial payments.
* Nodes already agree to do some overpayments to obfuscate the payment
amount. If the stuck payments are really small they could be considered as
overpayments (in case they eventually go through). This would work if the
sender sends overall `n+k` payments where `k` is the number of currently
stuck payments.

Potential Improvements (for better privacy)
================================
* Instead of using a sha-chain we could xor every preimage with a sequence
number making it harder for an attacker to correlate two consecutive
payments in the stream of payments via
```
r_0=rand()
H_0(r_0)
r_{i+1} = H_0 ^ (i+1)
H_{i+1} = H(r_{i+1})
```
Instead of a sequence number we could also do something like `(i+1)*x` (as
attackers should not be aware of the overall payment amount it will be hard
to guess that number). I guess you folks are aware of much better
cryptographic tricks to achieve what I suggest here. So please take this
just as an idea from a beginner in cryptography.

* If paths are reused for partial payments the sender should switch to a
preimage from a different sha-chain creating some path decorellation
* Even when going away from hashedpreimages when changing to MuSig and
multilocks a mechanism similar to HD-wallets should be usable for this
scheme

Sender Requirements
=================
The sender needs to keep track how many payments have been successfully
sent and how many are in transition / stuck.
For the privacy parts the sender additionally needs to watch out to cycle
through the shachains when reusing paths

Receiver Requirements
==================
More data needs to be stored / held in memory. (either `s` complete
sha-chains) or the `s` times the initial preimage of each chain and the
current payment hash. in the later case the computational overhead will be
increased.

Conclusion
=========
* With introduction of two new fields in BOLT 11 invoices and rather simple
code changes for invoice creation / preimage releasing we have the ability
to introduce flow and congestion control to multipath routing of payments.
* The proposed mechanism is not atomic. It is possible that a payee
receives only a fraction of all payments and stops collaborating
* Less HTLCs and liquidity will be bound in multipath routing schemes
resulting in a reduction of load for the network
* A few stuck payments can be neglected as "cost" of operation. While I
strongly support ideas from [0] we might not need them with this simple
trick.
* As I used the term `stream of payments` in the text this mechanism could
also be extended for a streaming payments protocol.

I am curios on your thoughts.

With kind regards Rene

[0]:
https://lists.linuxfoundation.org/pipermail/lightning-dev/2019-June/002029.html
Proposal
for Stuckless Payment by Hiroki
[1]:
https://lists.linuxfoundation.org/pipermail/lightning-dev/2018-February/000993.html
AMP:
Atomic Multi-Path Payments over Lightning by Conner & Laolu
[2]: https://arxiv.org/abs/1809.05088 Routing Cryptocurrency with the
Spider Network by Vibhaalakshmi et. al.

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190715/163114e7/attachment.html>;

📅 Original date posted:2019-06-25 📝 Original message: Hey ...

2023-06-09T12:55:16Z

In reply to nevent1q…08rg
_________________________

📅 Original date posted:2019-06-25
📝 Original message:
Hey Ugam,

I like the very clearly communicated idea and the fact that we can do crazy
stuff with the filler of the onions. I have two concerns / questions:

1.) In pathfinding we actually try to make payments smaller (like moving to
AMP) instead of combining payments. I think it was shown several times that
the probability of finding a path (successful route) decreases with larger
amounts. So saving fees might actually not be the metric that we are trying
to optimize.
2.) Am I correct that this proposal would only work with the spontaneous
payment scenario as the payment hashes of Eric and Grace could not just be
added up as easy as the preimages can to get the overall payment hash for
Alice? So in that sense on the invoice based system your proposal is not
working and we don't have a proof of payment as Alice already knows the
preimages? Could one resolve this in the world of scriptless scripts or
when changing to secret / curve point based preimages based on the discrete
log?

best Rene

On Tue, Jun 25, 2019 at 7:07 AM Ugam Kamat <ugamkamat1 at gmail.com> wrote:

> Hey guys,
>
>
>
> I’m kind of new to this mailing list, so let me know if this has been
> proposed previously. While reading Olaoluwa Osuntokun’s Spontaneous
> Payment proposal, I came up with the idea of simultaneous payments to
> multiple parties using the same partial route. In other words, say Alice,
> Bob, Charlie, Dave and Eric have channel opened with one another, and say
> Dave also has channel with Frank who has channel with Grace. Now, Alice is
> at a restaurant and wants to pay the bill amount to Eric (the restaurant
> owner) and a tip to Grace (who was her waiter). In the current scenario,
> Alice would have to send two payments A->B->C->D->E and A->B->C->D->F->G.
> However, if we repurpose the onion blob
> <https://github.com/ElementsProject/lightning/pull/2363>; in the same way
> as is needed for Spontaneous Payments, we can create a scenario where there
> is no path duplication. Dave would split the payments, one to Eric and
> other going to Grace through Frank. The preimage PM used in commitments
> A->B, B->C and C->D will be a function of pre-images P1 of D->E and P2 of
> D->F and F->G such that PM = f(P1, P2).
>
>
>
> *Proposal can be implemented by repurposing the onion in similar fashion
> as Spontaneous Payments with slight modification*
>
> This proposal works in similar fashion to Spontaneous Payment proposal, by
> packing in additional data in the unused hops. For B and C the onion blob
> will be identical to other lightning payments. When D parses the onion, the
> 4 MSB of the realm will tell D how much data can be extracted. This data
> will encode the hashes of the pre-images that would be used for commitment
> transaction towards Eric and other towards Frank. For simplicity and
> privacy, I propose using 2 onion blobs for the data. So the payload can be
> 64 + 33 bytes = 97 bytes. The first byte would indicate how many hashes are
> packed, so we have 96 bytes for the payload, meaning we can pack a maximum
> of 3 hashes for 3 route payments from D. Now D will split the onion (18
> hops as it has used the first two for bifurcation data) into number of
> routes. In the above case it will be 9 hops each. Now these two onions are
> similar to other lightning payments. The first hop tells D the
> short-channel id, amount to forward, CLTV and the padding. Since, the
> preimage is 32 bytes, we can pack that in one single hop that is received
> by the final party. This leaves the remaining 7 hops can be used for
> routing. Below figure depicts the onion split in terms of how A will create
> it. D will add the filler to make each onion have 20 hops. Onion data is
> encoded in the same order in which the payment hashes are packed in the
> bifurcation data for D.
>
>
>
> *Calculating the preimages*
>
> Eric and Grace will parse the onion and use the pre-images for settlement.
> Let P1 represent the pre-images of D->E and P2 of D->F and F->G. When the
> pre-images arrive at node D, it will combine them such that PM = f(P1, P2).
> The easiest way for both A and D to calculate that will be PM = SHA256(P1
> || P2 || ss_d). Where || represents concatenation and ss_d is the shared
> secret created using the ephemeral public key of sender (the one generated
> by Alice) and private key of Dave. The need for using shared secret is to
> prevent the vulnerability where one channel operator who has nodes across
> both branches can use them to calculate the PM. Using shared secret also
> ensures that it is in fact D that has parsed them together.
>
>
>
> *Advantages of this proposal:*
>
> - Commitment transactions between A & B, B & C, and C & D now carry
> only one HTLC instead of two
> - This means lower fees in case of on-chain settlement
> - Lower routing fees for Alice as Bob and Charlie would not get to
> charge for two routings
> - Since 483 is the max limit of the htlcs nodes can accepts,
> preventing duplication will allow more number of htlcs in flight.
> - If each payment of Eric and Grace is below the htlc min B or C
> accepts, but together if it is higher, this route is now usable
>
>
>
> *Some thoughts on if this proposal can be misused?*
>
> - The probability of transaction failures increases as now the
> transaction is dependent on 2/3 branches
>
>
>
> *Deployment*
>
> Not all nodes need to support this feature. For example, B, C, E, F, and
> G does not even know that the payment arrived through branching. The nodes
> that can handle branching of payments can signal that using global features.
>
>
>
>
>
> Ugam
>
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190625/c6415a59/attachment-0001.html>;
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 7836 bytes
Desc: not available
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190625/c6415a59/attachment-0001.png>;

📅 Original date posted:2019-03-29 📝 Original message: Dear ...

2023-06-09T12:54:39Z

In reply to nevent1q…g0r3
_________________________

📅 Original date posted:2019-03-29
📝 Original message:
Dear ZmnSCPxj and fellow lightning developers,

I want to point out two things and make a suggestion for a new gossip
message.

A good pruning heuristic is "channel capacity", which can be checked
> onchain (the value of the UTXO backing the channel is the channel capacity).
> It is good to keep channels with large capacity in the routemap, because
> such large channels are more likely to successfully route a payment than
> smaller channels.
> So it is reasonable to delete channels with low capacity when the routemap
> memory is becoming close to full.
>

Intuitively (without simulation). I encourage to make that process not
deerministic but rather probabilistic. It would be good if everyone had a
different set of channels. (which is somewhat achieved with everyone
keeping their local view)

Nodes still need to track their direct channels (so they are implicitly
> always in the routemap).
>

I strongly advice that the local view should be extended. Every node should
always track their friends of a friend network. Maybe we could even create
a new gossip query message `query_ask_egonetwork` that asks for the
egonetwork of a node (the egonetwork are all the direct friends of a node
together with their friendships) every node knows at least the nodes in
their ego network and over time also the edges between them.

If I was interested in my friend of a friend network I could just send the
`query_ask_egonetwork` message to all my peers.

Best Rene

But payee nodes making BOLT1 invoices could also provide `r` routes in the
> invoice, with the given routes being from nodes with high-capacity
> channels, so that even if the intermediate channels are pruned due to low
> capacity, it is possible to get paid.
>
> Regards,
> ZmnSCPxj
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190329/33d0bd13/attachment-0001.html>;

📅 Original date posted:2019-03-29 📝 Original message: Good ...

2023-06-09T12:54:39Z

In reply to nevent1q…0yer
_________________________

📅 Original date posted:2019-03-29
📝 Original message:
Good morning ZmnSCPxj,

Maybe I oversee something - in that case sorry for spamming the list - but
I don't understand how you could know the distance from yourself if you
don't know the entire topology? (unless u use it on the pruned view as you
suggested)

On the other hand querying for a certain depth would be possible with the
suggested `query ask egonetwork` in case for depth 3 one would have to peer
with the nodes from the friend of a friend network.

Best Rene

ZmnSCPxj <ZmnSCPxj at protonmail.com> schrieb am Fr., 29. März 2019, 09:47:

> Good morning Rene,
>
>
> Sent with ProtonMail Secure Email.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Friday, March 29, 2019 1:54 PM, René Pickhardt <
> r.pickhardt at googlemail.com> wrote:
>
> > Dear ZmnSCPxj and fellow lightning developers,
> >
> > I want to point out two things and make a suggestion for a new gossip
> message.
> >
> > > A good pruning heuristic is "channel capacity", which can be checked
> onchain (the value of the UTXO backing the channel is the channel capacity).
> > > It is good to keep channels with large capacity in the routemap,
> because such large channels are more likely to successfully route a payment
> than smaller channels.
> > > So it is reasonable to delete channels with low capacity when the
> routemap memory is becoming close to full.
> >
> > Intuitively (without simulation). I encourage to make that process not
> deerministic but rather probabilistic. It would be good if everyone had a
> different set of channels. (which is somewhat achieved with everyone
> keeping their local view)
>
> At a software engineer point-of-view, probabilistic can be difficult to
> test.
> This can be made deterministic by including an RNG seed in the input to
> this code.
>
> However, let me propose instead, in combination with your later thought:
>
> >
> > > Nodes still need to track their direct channels (so they are
> implicitly always in the routemap).
> >
> > I strongly advice that the local view should be extended. Every node
> should always track their friends of a friend network.
>
> Perhaps the pruning rule can be modified to include *distance from self*
> in addition to channel capacity.
> The nearer the channel is, the more likely it is retained.
> The further, the less likely.
> The larger the channel is, the more likely it is retained.
> The smaller, the less likely.
>
> The capacity divided by the distance can be used as a sorting key, and if
> pruning is needed, the smallest "score" is pruned until the routemap fits.
>
> This will lead to everyone having a different set of channels, while being
> likely to track their friend-of-friend network compared to more distant
> nodes.
>
> Of course, the pruning itself would affect the distance of the channel to
> the "self" node.
> So determinism may be difficult to achieve here anyway.
>
> Regards,
> ZmnSCPxj
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190329/54ccc4a4/attachment.html>;

📅 Original date posted:2019-03-20 📝 Original message: Hey ...

2023-06-09T12:54:37Z

In reply to nevent1q…gym4
_________________________

📅 Original date posted:2019-03-20
📝 Original message:
Hey everyone,

as you know I am just a mathematician and not a cryptographer by training
who is interested and amazed by the Lightning Network Protocol. I joined
this open source effort pretty late (in the beginning of 2018) and tried to
catch up ever since. At that time reading the BOLTs (despite the fact that
Rusty had his helpful blog articles) was really hard for me.

Over the past year - especially due to your help and open ears - I was able
to catch up by a lot. Yesterday I held a 4 hour workshop giving an
extensive overview about the Lightning Network and BOLT 1.0.

I decided to open source the slides to give something (hopefully useful)
back to the community. To the best of my knowledge this is the most
extensive overview of the BOLTs in form of slides and I also tried to
restructure the content in the hope of making it easier to approach fo
newbies like I was a year ago.

Please feel free to fork the slides on the google doc address:
https://docs.google.com/presentation/d/1-eyceLlSmcLpbPJLzj6_CnVYQdo1AUP3y5XD716U-Lg

I uploaded the pdf version of the slides at
https://commons.wikimedia.org/wiki/File:Introduction_to_the_Lightning_Network_Protocol_and_the_Basics_of_Lightning_Technology_(BOLT_aka_Lightning-rfc).pdf

Obviously I am very happy if you have suggestions how the slides could be
improved even further. Probably I will still have some mistakes with them.
But maybe you have more comments on a meta level about the structure or
detail level or visualizations...

Therefor I will be happy if you send back a pdf with comments. I also share
a commentable version with this mailinglist on google docs:
https://docs.google.com/presentation/d/1YCKZzE53xjnBndl3U0uy1-bHO4Y0rlAwgrnujzef1sY

I will present the slides next time on chaincodelabs lightning residency in
late june where - as far as I know - the talks will also be recorded. So
any feedback before then will be highly appreciated.

While this mail was note particularly about development I hope it is not
considered offtopic. In case you do so I apologize for any inconvenience.

With kind Regards Rene

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190320/7817d7be/attachment.html>;

📅 Original date posted:2019-03-15 📝 Original message: Hey ...

2023-06-09T12:54:35Z

In reply to nevent1q…jfpv
_________________________

📅 Original date posted:2019-03-15
📝 Original message:
Hey everyone,

during the spec meeting we have discussed intensively about dual funded
channels and potential game theory with the fees however I now believe that
we missed out another important crucial part which is the privacy of the
node providing liquidity.

While I have not seen a concrete example for a channel establishment
protocol that supports dual funded channels I have seen this proposal (
https://lists.linuxfoundation.org/pipermail/lightning-dev/2018-November/001532.html
) for advertising channel liquidity which assumed that the `open_channel`
message would be modified as follows:

`open_channel`:
new feature flag (channel_flags): option_liquidity_buy [2nd least
significant bit]
push_msat: set to fee payment for requested liquidity
[8 liquidity_msat_request]: (option_liquidity_buy) amount of dual funding
requested at channel open

...

If a node cannot provide the liquidity requested in `open_channel`, it must
return an error.

With such a protocol I could now (basically only at the cost of internet
traffic) probe a lower bound for the amount of BTC available by a node that
allows for dual funded channels and abort the channel establishing process
at some time before I ever spend / lock any of my own bitcoin.

As I can even participate in the peer protocol without having a single
channel open this situation seems to be even more severe.

I don't have a clear suggestion how to mitigate against this. One general
potential idea / solution would be to make spamming / probing more
expensive. For example we could require the person to open a channel first
and then ask the partner to splice something in (meaning we don't allow for
one tx dual funded channels). In that case the node requesting liquidity
had to do an onchain tx. also the requests to splice in can be identified
and the person who feels to be probed can choose to fail the channel. I am
not happy with my barrier as it would still be able to relatively cheaply
abuse this and we run into a whole bunch of game theory about fees again.

As the lightning network seems very keen to provide strong privacy to its
users (c.f.: onion routing, keeping channel balances private, encrypted
transport layer,...) I thought it is worthwhile pointing out the problem
with the privacy for dual funded channels even though I don't have a
concrete solution yet.

best Rene

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190315/6ff1782a/attachment.html>;

📅 Original date posted:2019-03-14 📝 Original message: Hey ...

2023-06-09T12:54:23Z

In reply to nevent1q…ltc4
_________________________

📅 Original date posted:2019-03-14
📝 Original message:
Hey everyone,

I am glad the suggestion is being picked up. At this time I want to respond
to two of the concerns that have been thrown in. I have some other comments
and ideas but would like to hold them back so that we can have more people
joining the discussion without bias also this mail will already be quite
long.

ZmnSCPxj suggested to introduce a `success_rate` for JIT routing. While
this success_rate can obviously only be estimated or configured I would
advice against including this to the protocol. As I mentioned before I
suggested to include JIT Routing as a MAY Recommendation so it is up to the
node to decide if it cannot earn `offered_fee_amount` to engage in the
JIT-rebalancing operation. A node operator might be willing in general to
to pay a fee for rebalancing even if there is not an outstanding routing
event taking place. So even while `rebalancing_fee_amount` >
`offered_fee_amount` the node could see the offered_fee_amount as a
discount for the planned rebalancing amount. We don't know that and I
honestly believe that the protocol should not make economical decisions for
the node. In any case rebalancing will overall increase the likelihood for
successful routing but it makes sense to defer the rebalancing operation to
a moment in which the liquidity is actually needed.

Regarding Ariels suggestion about reusing the payment hash with JIT Routing
I have some more thoughts:
Reusing the payment hash indeed seems like a good idea. However it produces
some technical issues which in my opinion can all be mitigated. it is just
a question with these challenges if it is worthwhile doing it?

I have drawn several situations and tried to construct an example in which
using the same payment hash for the JIT-rebalancing would result in a
severe problem with the payment process in the sense that it would be
compromised or somebody could steal funds. It could however be a privacy
issue as more nodes are being aware of the same payment (but that is also
the case with base-AMP)
I was not able to construct such a situation (under the assumption that the
rebalancing amount does not exceed the original payment amount). My feeling
(though I have not done this yet) is that one should be able to proof that
taking the same payment hash would always work and in fact create a
situation in which at least the rebalancing only takes place if the entire
payment was routed successfully.

Assuming someone will be able to proof that using the same payment hash for
JIT Routing is not an issue we still run into another problem (which I
believe can be fixed with another MUST rule but comes with quite some
implementation overhead.)

The deadlock problem when doing JIT Routing with the same payment hash:
When using the same payment hash there will be two htlc's (potentially of
different amounts) in opposing directions on the same channel (or in the
lnd case maybe between separate channels between the same two peers).
Unluckily without a novel rule this can produce a deadlock.

As an example take the situation from my initial email with an additional
recipient R1:

100 / 110 80 / 200 150/180
S ----------> B --------> R ----------> R1
\ /
80/200 \ / 100/200
\ /
T

Meaning we have the following channels:
S ---> B capacity: 110 A funds: 100 B funds: 10
B ---> R capacity: 200 B funds: 80 R funds: 120
B ---> T capacity: 200 B funds: 80 T funds: 120
T ---> R capacity: 200 T funds: 100 R funds: 100
R ---> R1 capacity: 180 R funds: 150 R1 funds: 30

neglecting fees the following htlcs would be offered
1.) S-->B amount: 90
2.) B-->T amount:50
3.) T-->R amount:50
4.) R-->B amount: 50
5.) B-->R amount: 90 (difficult to set up before 4. settles)
6.) R-->R1 amount: 90

while 1,5 and 6 are the original path 2,3,4 are the JIT rebalancing.

We see that in this situation using the same preimage results in a problem.
Since the rebalancing is not settled R will not accept the 5th htlc (B--->R
amount: 90) as there is not enough liquidity on B's side producing a
deadlock
However since the same payment hash is used it is save to combine the 4th
and 5th htlc to have the following situation:

1.) S-->B amount: 90
2.) B-->T amount:50
3.) T-->R amount:50
4.) R-->B will be removed or settled or replaced by the 5th htlcs with a
different amount (90 - 50)
5.) B-->R amount: 40
6.) R-->R1 amount: 90

Note that while theoretically it seems tempting to just have two htlc
outputs as the second node could always claim the htlc if the first claims
theirs. However this will not work onchain as potentially more funds are
spend than exist.

Therefor we need a MUST-rule to fix the deadlock problem (which could
probably also be formulated in a symmetric way):
If a node N offers an htlc to a partner with an amount x from whom the node
already received an htlc y (where y is smaller than x) the nodes must
create a new channel state discarding the old htlc but having a new offer
from N with the amount x-y.

This decreases the liquidity bound in the routing process, enables for a
critical channel to be rebalanced several times during several JIT
operations and keeps the onchain footprint low as there are less htlc
outputs.

Also as mentioned above it seems crucial that the rebalancing amount must
not exceed the original payment amount if the payment hash is being reused.
Imagine there was no R1 and R was actually the final destination and B
decides to rebalance an amount larger than necessary (which could not
happen in our setup). R could release the preimage before setting up the
final htlc from R back to B to fulfill the rebalancing request. This could
also happen if T was the final recipient (which R would not now!)

The only way how I see that this problem can be mitigated is by introducing
the following rule (morally even as a MUST rule)
If a node decides to engage in JIT Routing using the same payment hash as
the incoming htlc it SHOULD NOT rebalance an amount higher than the
incoming HTLCs. If it rebalances with a new payment hash it MAY use an
arbitrary amount.
Rational: (as described above)

Another problem while reusing the payment hash is that in this situation
the node who issued the invoice could be involved in a rebalancing act and
decline an htlc as the amount is not sufficient for the invoice. Therefor
we would have to set either the base-AMP feature flag or create a new one
for JIT-routing which would signal that there are more htlc's coming which
need to be combined to forward an onion.

In order to avoid this complex aggregation of htlcs we could also see this
process as a local AMP right a way saying that a node instead of forwarding
the onion MAY do a local base-AMP to the next recipient.

So in my opinion this suggestions to reuse the payment hash is not only
reasonable but actually desirable in particular if we can proof that it is
not a problem and if we agree that we can mitigate the technical challenges
which I described in this mail.

Best regards Rene

On Thu, Mar 14, 2019 at 2:08 PM Ariel Lorenzo-Luaces <arielluaces at gmail.com>
wrote:

> Hello Rene, ZmnSCPxj, and list
>
>
> I really like the proposal and I'm sure it's the correct way forward for
> reducing payment failures and increasing privacy (through mitigating
> probing based network analysis)
>
>
> However I am concerned that this proposal could introduce a vulnerability
> to a spoofed-payment attack.
>
>
> An adversary could create a malicious payment that's guaranteed to fail,
> for free. Any intermediary nodes on the path of the payment before it fails
> could lose fees due to rebalancing if the rebalancing path's success is not
> dependent on the original payment's success.
>
>
> Could anyone speak to this and confirm whether it would be possible for
> the rebalancing step to reuse the original payment hash? If this is
> possible then it should explicitly be included in this proposal.
>
>
> If reusing the payment hash is not possible on the routing path then JIT
> routing would need some other mitigation for the spoofed-payment attack.
>
>
> Cheers
>
>
> Ariel Lorenzo-Luaces
>
>
> On Mar 14, 2019, at 7:45 AM, ZmnSCPxj via Lightning-dev <
> lightning-dev at lists.linuxfoundation.org> wrote:
>>
>> Good morning Rene and list,
>>
>> Let us consider then the rule *when* a rebalancing would be beneficial to the node.
>>
>> The node is offered a fee amount (`offered_fee_amount`) for the forwarding.
>> It knows that, under current channel states, it will definitely have to fail and earn 0 fees.
>>
>> If it engages in JIT-routing, it must pay some fee (`rebalancing_fee_amount`) for the rebalancing operation.
>> But even if it successfully forwards its hop, it is still possible that the route will fail anyway and it will earn 0 fees.
>>
>> So let us consider the probability of success (`success_rate`), a value between 0 to 1.0.
>> This is the estimated probability that we will succeed the route after we forward it.
>>
>> We should only engage in JIT-routing if:
>>
>> offered_fee_amount * success_rate - rebalancing_fee_amount > 0
>>
>> The LHS of the subtraction is the expected earning, while the RHS of the subtraction is the expected cost.
>> The above is trivial accounting for determining net earnings.
>>
>> The fee amounts are trivially computable.
>> Presumably the rebalancing code can compute the fee given a particular rebalance path, and thus can provide `rebalancing_fee_amount`.
>>
>> The `success_rate` can be computed statically from some node data.
>> Better, is for the node to start with this precomputed static information, but augment this over time with its own experienced `success_rate` for all forwards.
>>
>> Regards,
>> ZmnSCPxj
>> ------------------------------
>>
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>>
>> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190314/35731641/attachment-0001.html>;

📅 Original date posted:2019-03-05 📝 Original message: Hey ...

2023-06-09T12:54:21Z

In reply to nevent1q…38vc
_________________________

📅 Original date posted:2019-03-05
📝 Original message:
Hey everyone,

In this mail I introduce the Just in Time Routing schema (aka JIT Routing).
Its main idea is to mitigate the disadvantages from our current source
based routing (i.e.: guessing a route that will work in the sense that it
has enough liquidity in each channel) and make the routing process a little
bit more like the best effort routing that we know from IP-forwarding. As
far as I know this will not decrease the privacy of the nodes. As part of
this Routing scheme nodes need to be able to quickly rebalance their
channels. Thus in this mail I also propose a heuristic for doing this
efficiently which I have implemented and seems to provide pretty good
results. Obviously the heuristic should be tested with the help of a
simulation. I did not have the chance to do that yet. Partly also because I
am lacking a proper dataset and I don't want to do this on artificial data.

The advantages of JIT Routing are:
* it is possible to do now without any protocol modification. In particular
no modifications of the onions are necessary.
* routing nodes can already easily implement it. By implementing it they
will increase the routing success even for nodes which are running older
implementations
* it seems to be logically equivalent to AMP Routing. In particular its
properties will also help base AMP once it is part of the protocol.
* local channel balance information along the route can now be part of the
path finding process while not decreasing the privacy by sharing
information about channel balances with others. In fact the privacy of
nodes is even being increased.

The disadvantages seem:
* it might economically not be incentivized for a routing node in every
situation. Theoretically it can even happen that a node pays a fee in order
to use this technique but can't earn the routing fee as the onion fails
later. Nodes can implement risk management strategies to mitigate this
issue.
* The routing process might take a longer time as it starts sub routing
processes.
* While doing JIT routing the capacity for channels should be reserved even
before HTLCs are set up (to prevent hostile recursive chains of rebalancing
operations)

Obviously routing single big payments is a challenge for the lightning
network. During the developer summit in Adelaide we have agreed to put Base
AMP to BOLT 1.1. To review Base AMP the idea is basically after receiving a
payment hash to create several onions on various routes to the recipient.
While Base AMP in theory can find the maxflow / min cut and achieve maximum
liquidity it is not clear yet how well Base AMP will really work.

While it has been shown that smaller payments have a higher chance to be
routed successfully there is the downside that we have more payments which
increases the likelihood that any one of those payment eventually fails. As
far as I know there have not been any studies researching this fact. Also
the fact remains that Base AMP is still a source base routing protocol
putting the sender into a tough spot as it has to guess which routes might
work.

How to JIT Routing?

For the BOLTs we basically need one Recommendation (in fact even today
nodes can do this without this explicit recommendation, but I would suggest
to add the recommendation):

If a node cannot forward an incoming HTLC because the node has not enough
funds on the outgoing channel the node MAY pause the routing process and
try to rebalance the channel that misses liquidity. If it isn't able to
rebalance the channel it should fail the onion sending back an insufficient
wire funds error `temporary_channel_failure`

Let us consider the following Graph and situation for an example:

100 / 110 80 / 200
S ----------> B --------> R
\ /
80/200 \ / 100/200
\ /
T

Meaning we have the following channels:
S ---> B capacity: 110 A funds: 100 B funds: 10
B ---> R capacity: 200 B funds: 80 R funds: 120
B ---> T capacity: 200 B funds: 80 T funds: 120
T ---> R capacity: 200 T funds: 100 R funds: 100

Let us assume S wants to send a payment of 90 to R. With this distribution
of funds this will not work with a single route S ---> B ---> R as the
channel B--->R can only forward 78 (taking the channel reserve of 1% into
consideration)

Now with Base AMP after a protocol update this would be resolved by making
two onions forwarding for example 45 each. Also S would have to pay more
routing fees as the basefee of B will be charged twice.

Onion 1: S ---> B ---> R
Onion 2: S ---> B ---> T ---> R

However it is S again who has to guess how the problems with the liquidity
are it does not know how B has spread its funds between R and T (and
potentially other channels)

With the above recommendation in place B can create a different onion with
lets say moving 50 from the B---> T channel to the B ---> R channel
resulting in the following situation:

100 / 110 130 / 200
S ----------> B --------> R
\ /
30/200 \ / 50/200
\ /
T

Meaning we have the following channels:
S ---> B capacity: 110 A funds: 100 B funds: 10
B ---> R capacity: 200 B funds: 130 R funds: 70
B ---> T capacity: 200 B funds: 30 T funds: 170
T ---> R capacity: 200 T funds: 50 R funds: 150

Now B can easily pass the original onion with a value of 90 which was
designed for the route S ---> B ---> R

Obviously there can also be issues when B tries to rebalance their channels
as the attempted event of rebalancing might trigger another JIT operation
at another node (potentially making a later stage of the original onion
harder to be forwarded). Also B does not have full information about the
T---> R channel. Yet B has more information about B's neighbourhood than S
does as B knows which channels are liquid and how many paths exist between
the endpoints of those channels. This B should be able to make a more
educated decision as the originator node S. Since B would only do a
rebalancing if the routing fees for the rebalancing are cheaper than the
fees they collect (and other nodes would do the same) the risk for and
infinit cascade of rebalancing operations should be mitigated. Also the
total CLTV for the rebalancing operation should be smaller than the
original onion. A sender of the onion could obviously start with larger
cltv_deltas to allow routing nodes to have some buffer. This is also good
in the sense of shadow routing as described in the BOLTs.

Another problem that might arise is the question if the routing node is
able to quickly compute rebalancing paths. I have been working on a
c-lightning plugin for rebalancing over the weekend (it is actually when I
came up with the idea of JIT routing). Currently I use the following
promising heuristic for rebalancing:
I look at the friend of a friend network of the node that wants to
rebalance channels. This network consists of alle channel partners of the
node and their channels. This will in particular have all triangles and
rectangles of the lightning network that the node that wants to rebalance
is part of. Now I remove the node that wants to rebalance from the friend
of the friend network. The resulting graph should always stay pretty small.
In a regular small world network the friend of a friend graph consists of
roughly 10k nodes. In particular after pruning nodes with degree 1 (which
can't be used to rebalance on this subgraph) we have a pretty small
network. After extracting this subnetwork (which for general operation
could always be maintained while gossip messages are coming in) the
computation to suggest several hundred rebalancing cycles and even ordering
them by price takes less than 1 second on my machine.
So far I have been pretty successful finding several (cheap!) rebalancing
suggestions in this network from my lightning node (which has about 40
channels)

I will release the code for the rebalancing schema soon but I wanted to
have the idea already out in order to get more feedback while working on
this.

best Rene

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190305/8ed84b9d/attachment.html>;

📅 Original date posted:2019-01-28 📝 Original message: Dear ...

2023-06-09T12:54:06Z

In reply to nevent1q…hhzg
_________________________

📅 Original date posted:2019-01-28
📝 Original message:
Dear Melvin,

I believe the scheme lightning: should only apply
* for payments in the form of bolt11 strings
* to identifiy nodes like lightning:node_id at ipaddr:port
* maybe to identify channels (look up the short_channel_id of the form of a
triple separated by the letterx. BLOCKHEIGHTxTXINDEXxOUTPUTINDEX)

the orther addresses you mentioned already have a sceme e.g. tcp:

keep in mind that the ip address and port may change but the only real
identifier is the node_id which as you mentioned correctly is the pubkey
from an HD seed and it is bech32 encoded. Also the short channel id points
to a funding transaction on the base layer so implicitly to identify the
base layer transaction we do also need the SHA-256 hash of the genesis
block otherwise it is not clear that the blockheigt, txindex, output index
triple belongs specifically to the bitcoin blockchain.

with kind regards Rene

On Mon, Jan 28, 2019 at 7:13 AM Melvin Carvalho <melvincarvalho at gmail.com>
wrote:

>
>
> On Thu, 24 Jan 2019 at 13:42, René Pickhardt <r.pickhardt at googlemail.com>
> wrote:
>
>> Dear Melvin,
>>
>> I believe the vocabulary is not consistent across implementations. For
>> example if you look at c lightning there is no such command `describegraph`
>> but there are the two commands `listnodes` and `listchannels` which should
>> give the same information. For both I have attached a sample output at the
>> end of the mail to demonstrate how the naming of the vocabulary differs.
>>
>> Since the data of these commands is taken from the gossip store which
>> stores the gossip messages I would suggest to take the vocabulary from the
>> BOLT 07 which defines the gossip messages. Also this mailinglist is for
>> protocol development and the spec should be the authorative source for
>> naming:
>> https://github.com/lightningnetwork/lightning-rfc/blob/master/07-routing-gossip.md
>>
>> There are more terms specified in other bolts which could be a basis for
>> a vocabulary. I have created an overview by hand and made a Pull Request to
>> the repository which has not been merged yet as it was the wish of the
>> developers to have such a list to be extracted automatically.
>> Still this the overview in that pull request could serve as a basis for
>> such a vocabulary :
>> https://github.com/lightningnetwork/lightning-rfc/pull/458
>>
>> Now the example outputs from c-lightning we already that there are
>> differences in naming. Take the identifier for a node for example:
>> * BOLT07: node_id
>> * LND: pub_key
>> * clightning: nodeid
>>
>> example outputs:
>>
>> lighting-cli listnodes
>> {
>> "nodeid":
>> "02396bf51e81f8f67eaca3652271b4fe8d3f57bedb9578af711606391c5c66760e",
>> "alias": "PuraSloboda",
>> "color": "68f442",
>> "last_timestamp": 1548200218,
>> "globalfeatures": "",
>> "global_features": "",
>> "addresses": [
>> {
>> "type": "ipv4",
>> "address": "144.136.223.22",
>> "port": 9735
>> }
>> ]
>> }
>>
>> lightning-cli listchannels
>> {
>> "source":
>> "03bb88ccc444534da7b5b64b4f7b15e1eccb18e102db0e400d4b9cfe93763aa26d",
>> "destination":
>> "0272045af48b9871013753f7cce1cf82ed80b97d669ca44709e01976a67df80adc",
>> "short_channel_id": "559893:1912:0",
>> "public": true,
>> "satoshis": 47000,
>> "message_flags": 0,
>> "channel_flags": 1,
>> "flags": 1,
>> "active": true,
>> "last_update": 1548332847,
>> "base_fee_millisatoshi": 1000,
>> "fee_per_millionth": 1,
>> "delay": 144
>> }
>>
>> With kind regards Rene
>>
>
> This is extremely helpful, thank you!
>
> I will go with the RFC naming then. I'm starting with Nodes and Edges and
> will put together a document and demo for review. First step is to do
> Nodes.
>
> I have two questions
>
> 1. node_id -- that's basically your public key -- what types of key and
> serialization is this (my guess an ecdsa pub key derived from the HD seed),
> any pointers would be great
>
> 2. regarding the four address types :
>
> - 1: ipv4; data = [4:ipv4_addr][2:port] (length 6)
> - 2: ipv6; data = [16:ipv6_addr][2:port] (length 18)
> - 3: Tor v2 onion service; data = [10:onion_addr][2:port] (length 12)
> - version 2 onion service addresses; Encodes an 80-bit, truncated
> SHA-1 hash of a 1024-bit RSA public key for the onion service
> (a.k.a. Tor hidden service).
> - 4: Tor v3 onion service; data = [35:onion_addr][2:port] (length 37)
> - version 3 (prop224
> <https://gitweb.torproject.org/torspec.git/tree/proposals/224-rend-spec-ng.txt>;)
> onion service addresses; Encodes: [32:32_byte_ed25519_pubkey] ||
> [2:checksum] || [1:version], where checksum = sha3(".onion
> checksum" | pubkey || version)[:2].
>
>
> These can be looked up in the spec. But in the semantic web we like,
> where possible, to have self describing data. In particular we like URIs
> with a scheme or protocol so instead of example.com we'll have
> http://example.com
>
> In this context would the scheme be lightning: for all 4 address types, or
> is that just for bolt11?
>
>
>
>>
>> On Thu, Jan 24, 2019 at 1:21 PM Melvin Carvalho <melvincarvalho at gmail.com>
>> wrote:
>>
>>>
>>>
>>> On Mon, 21 Jan 2019 at 14:11, René Pickhardt <r.pickhardt at googlemail.com>
>>> wrote:
>>>
>>>> Dear Melvin,
>>>>
>>>> have you looked into the W3C Payment Group?
>>>> https://www.w3.org/TR/payment-request/ The entire field of semantic
>>>> web kind of originated from W3C and they are working on a recommendation
>>>> for browser vendors to enable a low level payment API.
>>>>
>>>> Also there is LightningJoule that builds on top of webln. While this is
>>>> not an otology it goes implicitly in a similar direction (c.f.:
>>>> https://github.com/wbobeirne/webln and in particular this discussion:
>>>> https://github.com/wbobeirne/webln/issues/1 in which Will said that in
>>>> his thoughts webln is different to the W3C Payment Group.)
>>>>
>>>> I am looking forward to see your progress with integrating Lightning to
>>>> the semantic web!
>>>>
>>>> with kind regards Rene
>>>>
>>>
>>> My first observation is these two data structures in lnd describe graph,
>>> one for channels and one for nodes. These seem to be two fundamental
>>> concepts in lightning.
>>>
>>> Channel
>>>
>>> {
>>> "channel_id": "615605565348708353",
>>> "chan_point":
>>> "d8cfed73e0004fe1427d3045c5b20da0418f3cb803e8e35be48ee713aadbf56d:1",
>>> "last_update": 1548330355,
>>> "node1_pub":
>>> "024a2e265cd66066b78a788ae615acdc84b5b0dec9efac36d7ac87513015eaf6ed",
>>> "node2_pub":
>>> "03e03c56bb540c36b9e77c2aea2bb6529b907ece6c1395228c05459af13d0e2a5c",
>>> "capacity": "1000000",
>>> "node1_policy": {
>>> "time_lock_delta": 144,
>>> "min_htlc": "1000",
>>> "fee_base_msat": "1000",
>>> "fee_rate_milli_msat": "1",
>>> "disabled": false
>>> },
>>> "node2_policy": {
>>> "time_lock_delta": 144,
>>> "min_htlc": "1000",
>>> "fee_base_msat": "1000",
>>> "fee_rate_milli_msat": "1",
>>> "disabled": false
>>> }
>>> }
>>>
>>> Node
>>>
>>> {
>>> "last_update": 1547380072,
>>> "pub_key":
>>> "0200072fd301cb4a680f26d87c28b705ccd6a1d5b00f1b5efd7fe5f998f1bbb1f1",
>>> "alias": "OutaSpace",
>>> "addresses": [
>>> {
>>> "network": "tcp",
>>> "addr": "46.163.78.93:9760"
>>> },
>>> {
>>> "network": "tcp",
>>> "addr": "[2a01:488:66:1000:2ea3:4e5d:0:1]:9760"
>>> },
>>> {
>>> "network": "tcp",
>>> "addr": "2dkobxxunnjatyph.onion:9760"
>>> },
>>> {
>>> "network": "tcp",
>>> "addr":
>>> "nzslu33ecbokyn32teza2peiiiuye43ftom7jvnuhsxdbg3vhw7w3aqd.onion:9760"
>>> }
>>> ],
>>> "color": "#123456"
>>> },
>>>
>>> It would be useful to write a vocab for these and then document what
>>> they mean. It would then be possible to add markup to an explorer to make
>>> it self documenting.
>>>
>>> My first question is : are these terms consistent across different
>>> implementations e.g. c-lightning, eclair ?
>>>
>>>
>>>>
>>>>
>>>> On Mon, Jan 21, 2019 at 7:17 AM Melvin Carvalho <
>>>> melvincarvalho at gmail.com> wrote:
>>>>
>>>>> Hi All
>>>>>
>>>>> I work on the solid project [1] and am very interested in the
>>>>> lightning network.
>>>>>
>>>>> In particular, I am looking at trying to create an integration between
>>>>> lightning (layer 2) and solid (layer 3? web layer?).
>>>>>
>>>>> The first step towards integration would be to port some of the
>>>>> lightning concepts to the semantic web. This is done by creating an
>>>>> ontology.
>>>>>
>>>>> Does anyone know of any existing work in this area. Alternatively,
>>>>> does anyone have an interest to collaborate on an ontology?
>>>>>
>>>>> Best
>>>>> Melvin
>>>>>
>>>>> [1] https://solid.mit.edu/
>>>>> _______________________________________________
>>>>> Lightning-dev mailing list
>>>>> Lightning-dev at lists.linuxfoundation.org
>>>>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>>>>>
>>>>
>>>>
>>>> --
>>>> https://www.rene-pickhardt.de
>>>>
>>>> Skype: rene.pickhardt
>>>>
>>>> mobile: +49 (0)176 5762 3618
>>>>
>>>
>>
>> --
>> https://www.rene-pickhardt.de
>>
>> Skype: rene.pickhardt
>>
>> mobile: +49 (0)176 5762 3618
>>
>

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190128/f3c7bdb1/attachment-0001.html>;

📅 Original date posted:2019-01-24 📝 Original message: Dear ...

2023-06-09T12:54:05Z

In reply to nevent1q…3ztq
_________________________

📅 Original date posted:2019-01-24
📝 Original message:
Dear Melvin,

I believe the vocabulary is not consistent across implementations. For
example if you look at c lightning there is no such command `describegraph`
but there are the two commands `listnodes` and `listchannels` which should
give the same information. For both I have attached a sample output at the
end of the mail to demonstrate how the naming of the vocabulary differs.

Since the data of these commands is taken from the gossip store which
stores the gossip messages I would suggest to take the vocabulary from the
BOLT 07 which defines the gossip messages. Also this mailinglist is for
protocol development and the spec should be the authorative source for
naming:
https://github.com/lightningnetwork/lightning-rfc/blob/master/07-routing-gossip.md

There are more terms specified in other bolts which could be a basis for a
vocabulary. I have created an overview by hand and made a Pull Request to
the repository which has not been merged yet as it was the wish of the
developers to have such a list to be extracted automatically.
Still this the overview in that pull request could serve as a basis for
such a vocabulary :
https://github.com/lightningnetwork/lightning-rfc/pull/458

Now the example outputs from c-lightning we already that there are
differences in naming. Take the identifier for a node for example:
* BOLT07: node_id
* LND: pub_key
* clightning: nodeid

example outputs:

lighting-cli listnodes
{
"nodeid":
"02396bf51e81f8f67eaca3652271b4fe8d3f57bedb9578af711606391c5c66760e",
"alias": "PuraSloboda",
"color": "68f442",
"last_timestamp": 1548200218,
"globalfeatures": "",
"global_features": "",
"addresses": [
{
"type": "ipv4",
"address": "144.136.223.22",
"port": 9735
}
]
}

lightning-cli listchannels
{
"source":
"03bb88ccc444534da7b5b64b4f7b15e1eccb18e102db0e400d4b9cfe93763aa26d",
"destination":
"0272045af48b9871013753f7cce1cf82ed80b97d669ca44709e01976a67df80adc",
"short_channel_id": "559893:1912:0",
"public": true,
"satoshis": 47000,
"message_flags": 0,
"channel_flags": 1,
"flags": 1,
"active": true,
"last_update": 1548332847,
"base_fee_millisatoshi": 1000,
"fee_per_millionth": 1,
"delay": 144
}

With kind regards Rene

On Thu, Jan 24, 2019 at 1:21 PM Melvin Carvalho <melvincarvalho at gmail.com>
wrote:

>
>
> On Mon, 21 Jan 2019 at 14:11, René Pickhardt <r.pickhardt at googlemail.com>
> wrote:
>
>> Dear Melvin,
>>
>> have you looked into the W3C Payment Group?
>> https://www.w3.org/TR/payment-request/ The entire field of semantic web
>> kind of originated from W3C and they are working on a recommendation for
>> browser vendors to enable a low level payment API.
>>
>> Also there is LightningJoule that builds on top of webln. While this is
>> not an otology it goes implicitly in a similar direction (c.f.:
>> https://github.com/wbobeirne/webln and in particular this discussion:
>> https://github.com/wbobeirne/webln/issues/1 in which Will said that in
>> his thoughts webln is different to the W3C Payment Group.)
>>
>> I am looking forward to see your progress with integrating Lightning to
>> the semantic web!
>>
>> with kind regards Rene
>>
>
> My first observation is these two data structures in lnd describe graph,
> one for channels and one for nodes. These seem to be two fundamental
> concepts in lightning.
>
> Channel
>
> {
> "channel_id": "615605565348708353",
> "chan_point":
> "d8cfed73e0004fe1427d3045c5b20da0418f3cb803e8e35be48ee713aadbf56d:1",
> "last_update": 1548330355,
> "node1_pub":
> "024a2e265cd66066b78a788ae615acdc84b5b0dec9efac36d7ac87513015eaf6ed",
> "node2_pub":
> "03e03c56bb540c36b9e77c2aea2bb6529b907ece6c1395228c05459af13d0e2a5c",
> "capacity": "1000000",
> "node1_policy": {
> "time_lock_delta": 144,
> "min_htlc": "1000",
> "fee_base_msat": "1000",
> "fee_rate_milli_msat": "1",
> "disabled": false
> },
> "node2_policy": {
> "time_lock_delta": 144,
> "min_htlc": "1000",
> "fee_base_msat": "1000",
> "fee_rate_milli_msat": "1",
> "disabled": false
> }
> }
>
> Node
>
> {
> "last_update": 1547380072,
> "pub_key":
> "0200072fd301cb4a680f26d87c28b705ccd6a1d5b00f1b5efd7fe5f998f1bbb1f1",
> "alias": "OutaSpace",
> "addresses": [
> {
> "network": "tcp",
> "addr": "46.163.78.93:9760"
> },
> {
> "network": "tcp",
> "addr": "[2a01:488:66:1000:2ea3:4e5d:0:1]:9760"
> },
> {
> "network": "tcp",
> "addr": "2dkobxxunnjatyph.onion:9760"
> },
> {
> "network": "tcp",
> "addr":
> "nzslu33ecbokyn32teza2peiiiuye43ftom7jvnuhsxdbg3vhw7w3aqd.onion:9760"
> }
> ],
> "color": "#123456"
> },
>
> It would be useful to write a vocab for these and then document what they
> mean. It would then be possible to add markup to an explorer to make it
> self documenting.
>
> My first question is : are these terms consistent across different
> implementations e.g. c-lightning, eclair ?
>
>
>>
>>
>> On Mon, Jan 21, 2019 at 7:17 AM Melvin Carvalho <melvincarvalho at gmail.com>
>> wrote:
>>
>>> Hi All
>>>
>>> I work on the solid project [1] and am very interested in the lightning
>>> network.
>>>
>>> In particular, I am looking at trying to create an integration between
>>> lightning (layer 2) and solid (layer 3? web layer?).
>>>
>>> The first step towards integration would be to port some of the
>>> lightning concepts to the semantic web. This is done by creating an
>>> ontology.
>>>
>>> Does anyone know of any existing work in this area. Alternatively, does
>>> anyone have an interest to collaborate on an ontology?
>>>
>>> Best
>>> Melvin
>>>
>>> [1] https://solid.mit.edu/
>>> _______________________________________________
>>> Lightning-dev mailing list
>>> Lightning-dev at lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>>>
>>
>>
>> --
>> https://www.rene-pickhardt.de
>>
>> Skype: rene.pickhardt
>>
>> mobile: +49 (0)176 5762 3618
>>
>

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190124/8007367f/attachment-0001.html>;

📅 Original date posted:2019-01-21 📝 Original message: Dear ...

2023-06-09T12:54:04Z

In reply to nevent1q…fpfp
_________________________

📅 Original date posted:2019-01-21
📝 Original message:
Dear Melvin,

have you looked into the W3C Payment Group?
https://www.w3.org/TR/payment-request/ The entire field of semantic web
kind of originated from W3C and they are working on a recommendation for
browser vendors to enable a low level payment API.

Also there is LightningJoule that builds on top of webln. While this is not
an otology it goes implicitly in a similar direction (c.f.:
https://github.com/wbobeirne/webln and in particular this discussion:
https://github.com/wbobeirne/webln/issues/1 in which Will said that in his
thoughts webln is different to the W3C Payment Group.)

I am looking forward to see your progress with integrating Lightning to the
semantic web!

with kind regards Rene

On Mon, Jan 21, 2019 at 7:17 AM Melvin Carvalho <melvincarvalho at gmail.com>
wrote:

> Hi All
>
> I work on the solid project [1] and am very interested in the lightning
> network.
>
> In particular, I am looking at trying to create an integration between
> lightning (layer 2) and solid (layer 3? web layer?).
>
> The first step towards integration would be to port some of the lightning
> concepts to the semantic web. This is done by creating an ontology.
>
> Does anyone know of any existing work in this area. Alternatively, does
> anyone have an interest to collaborate on an ontology?
>
> Best
> Melvin
>
> [1] https://solid.mit.edu/
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190121/bc0c5b29/attachment.html>;

📅 Original date posted:2019-01-22 📝 Original message: Hey ...

2023-06-09T12:54:02Z

In reply to nevent1q…mymy
_________________________

📅 Original date posted:2019-01-22
📝 Original message:
Hey Alex,

I think this is currently not being implemented. Also there is a
mailinglist particularly for issues related to c-lightning at:
https://lists.ozlabs.org/listinfo/c-lightning and of course issues like
this could also be asked in the bug tracker at:
https://github.com/ElementsProject/lightning

with kind regards Rene

On Tue, Jan 22, 2019 at 1:08 PM Alex P <ap at coinomat.com> wrote:

> Hello guys!
>
> Is there a way to bind RPC to IP:port, not to unix socket?
>
> Of course, it's easy to patch source code, but may be there is a param
> for config?
>
> Thanks!
>
>
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190122/315fa6cf/attachment.html>;

📅 Original date posted:2018-11-29 📝 Original message: Hey ...

2023-06-09T12:53:14Z

In reply to nevent1q…tysp
_________________________

📅 Original date posted:2018-11-29
📝 Original message:
Hey CJP,

I am still not 100% through the SPHINX paper so it would be great if at
least another pair of eyes could lookt at this. However from the original
SPHINX paper I quote:

"Besides extracting the shared key, each mix has to be provided with
authentic and confidential routing information to direct the message to the
subsequent mix, or to its final destination. We achieve this by a simple
encrypt-then-MAC mechanism. A secure stream cipher or AES in counter mode
is used for encryption, and a secure MAC (with some strong but standard
properties) is used to ensure no part of the message header containing
routing information has been modified. Some padding has to be added at each
mix stage, in order to keep the length of the message invariant at each
hop."

At first I thought this would mean that the HMAC ensures that the previous
hop cannot change the routing information. which was the first answer that
I wanted to give. However I am confused now too. The HMAC commits to the
next onion. So if the entire onion was exchanged and a new HMAC was
provided (as you suggest) the processing hop would not know this. Such a
use case would obviously lead to a routing scenario which would not succeed
and would hardly be useful (unless the previous hop plans a reverse dos
attacks from error messages or some other sabotage attacks which are
references in the SPHINX paper but not discussed explicitly).

On a second thought I reviewed chapter 2.1 of the Sphinx paper in which the
thread model for attackers is described. As far as I understand that
section one attack vector for which the HMAC shall help are man in the
middle attacks. If HMACs are being used some bitflipping by man in the
middles would be detected. However I think if a man in the middle speaks
the BOLT protocol they could exchange the entire package and provide a new
HMAC as a previous hop could do. Also the Thread model does only speak
about security of the message not so much about the reliability of the
protocol. I believe it is quite clear that if a routing node wants to
manipulate the onion they can do so. In the same way how they can decide
not to forward the onion.

--> So the mix network itself can make sure that no wrong messages are
delivered it cannot make sure that messages (which are unseen and unknown
from where they came) are intercepted.

Besides the Bitflipping usecase that I mentioned I agree with your
criticism and also don't see the necessity of the HMAC anymore. The message
is encrypted anyway and if bits are flipped the decrypted version will just
be badly formated. If the header was manipulated the next hop would not be
able to decrypt.

Best regards Rene

Am Do., 29. Nov. 2018, 16:31 hat Corné Plooy via Lightning-dev <
lightning-dev at lists.linuxfoundation.org> geschrieben:

> Hi,
>
>
> Is there a reason why we have HMACs in Sphinx? What could go wrong if we
> didn't?
>
> A receiving node doesn't know anyway what the origin node is; I don't
> see any attack mode where an attacker wouldn't be able to generate a
> valid HMAC.
>
> A receiving node only knows which peer sent it a Sphinx packet;
> verification that this peer really sent this Sphinx packet is (I think)
> already done on a lower protocol layer.
>
>
> AFAICS, The only real use case of the HMAC value is the special case of
> a 0-valued HMAC, indicating the end of the route. But that's just silly:
> it's essentially a boolean, not any kind of cryptographic verification.
>
>
> CJP
>
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20181129/ea78ec4f/attachment.html>;

📅 Original date posted:2018-11-23 📝 Original message: Hey ...

2023-06-09T12:53:06Z

In reply to nevent1q…jjq0
_________________________

📅 Original date posted:2018-11-23
📝 Original message:
Hey Cezary,

sorry I misread your initial question. I thought you where referring to the
(probably bigger problem) of getting the commitment transaction to be mined
because RBF does not work. But if we assume that your channel partner
published an outdated commitment transaction which got mined then you can
claim (both) outputs (your node should do this automatically) with this
penalty transaction. This penalty transaction is spending the outputs of
the commitment transaction and is signed with your nodes private key.
Therefor as far as I know you should be able to RBF this penalty
transaction. Also I believe you understand the process correctly.

Actually since the timelock on the commitment transaction will at some
point in time be over (in which case also your channel partner can spend
their output) you have an economic incentive to quickly get the penalty
transaction minded by using rather high fees or in case at this time really
a lot of transactions come in to RBF. I currently see no reason why you
could not RBF the penalty transaction. In case I oversee something I am
sure someone here on the list will correct me.

best regards Rene

On Fri, Nov 23, 2018 at 8:34 PM Cezary Dziemian <cezary.dziemian at gmail.com>
wrote:

> Thanks for answer,
>
> My knowledge is mostly based on this article:
>
>
> https://bitcoinmagazine.com/articles/understanding-the-lightning-network-part-building-a-bidirectional-payment-channel-1464710791/
>
> Graph at the end shows that in order to claim former channel partner funds
> I need to provide child transaction that contains my signature and secret.
> This secret is evidence.that partner didn't commit the last transaction.
>
> So the penalty transaction uses comitment transaction output as its input
> and penalty transaction can be sign by one side only. Am I right, or I just
> don't understand how it works? Or maybe this graph do not represents
> correctly how commitment and penalty transactions are already developed?
>
> Best Regards,
> Cezary Dziemian
>
>
> pt., 23 lis 2018 o 19:07 René Pickhardt <r.pickhardt at googlemail.com>
> napisał(a):
>
>> Dear Cezary,
>>
>> as far as I understand the problem in the case of a unilateral (force)
>> close are:
>>
>> 1.) In order to RBF your commitment transaction you would have to have
>> the signature of your former channel partner. since you initiated a force
>> close it is unlikely that you get this signature to RBF because then you
>> could have done a mutual close right away which is cheaper since less tx
>> are invovled to claim all funds back.
>> 2.) In order to CPFP you have to be able to spend your output which can't
>> work because there is a timelock on it.
>>
>> I believe on the last lightning developer summit this issue was discussed
>> and it was agreed that for BOLT1.1 we want have a third output in the
>> commitment transactions which anyone can spend (OP_TRUE) and which is just
>> above the dust level. This output is supposed to have no timelock so that
>> anyone can CPFP it. In the general case miners of the block could collect
>> the output as a fee. You can find a pointer to this on this wikipage in the
>> lightning-rfc git repo:
>> https://github.com/lightningnetwork/lightning-rfc/wiki/Lightning-Specification-1.1-Proposal-States
>> (look in the section tx and fees)
>>
>> best Rene
>>
>> On Fri, Nov 23, 2018 at 6:30 PM Cezary Dziemian <
>> cezary.dziemian at gmail.com> wrote:
>>
>>> Hello all,
>>>
>>> Sorry for my ignorance. I have two questions related with penalty txs. I
>>> assume, that when someone commits obsolete commitment tx, my node
>>> automatically commit penalty transaction.
>>>
>>> What if fees suddenly increases? Can my node use RBF to increase fee?
>>>
>>> Is there any approach common to major 3 implementations?
>>>
>>> How much time (how many blocks) do my node have to commit penalty tx? Is
>>> there some value common for implementations?
>>>
>>> Best regards,
>>> Cezary Dziemian
>>> _______________________________________________
>>> Lightning-dev mailing list
>>> Lightning-dev at lists.linuxfoundation.org
>>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>>>
>>
>>
>> --
>> https://www.rene-pickhardt.de
>>
>> Skype: rene.pickhardt
>>
>> mobile: +49 (0)176 5762 3618
>>
>

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20181123/bb27a317/attachment-0001.html>;

📅 Original date posted:2018-11-23 📝 Original message: Dear ...

2023-06-09T12:53:05Z

In reply to nevent1q…uc7w
_________________________

📅 Original date posted:2018-11-23
📝 Original message:
Dear Cezary,

as far as I understand the problem in the case of a unilateral (force)
close are:

1.) In order to RBF your commitment transaction you would have to have the
signature of your former channel partner. since you initiated a force close
it is unlikely that you get this signature to RBF because then you could
have done a mutual close right away which is cheaper since less tx are
invovled to claim all funds back.
2.) In order to CPFP you have to be able to spend your output which can't
work because there is a timelock on it.

I believe on the last lightning developer summit this issue was discussed
and it was agreed that for BOLT1.1 we want have a third output in the
commitment transactions which anyone can spend (OP_TRUE) and which is just
above the dust level. This output is supposed to have no timelock so that
anyone can CPFP it. In the general case miners of the block could collect
the output as a fee. You can find a pointer to this on this wikipage in the
lightning-rfc git repo:
https://github.com/lightningnetwork/lightning-rfc/wiki/Lightning-Specification-1.1-Proposal-States
(look in the section tx and fees)

best Rene

On Fri, Nov 23, 2018 at 6:30 PM Cezary Dziemian <cezary.dziemian at gmail.com>
wrote:

> Hello all,
>
> Sorry for my ignorance. I have two questions related with penalty txs. I
> assume, that when someone commits obsolete commitment tx, my node
> automatically commit penalty transaction.
>
> What if fees suddenly increases? Can my node use RBF to increase fee?
>
> Is there any approach common to major 3 implementations?
>
> How much time (how many blocks) do my node have to commit penalty tx? Is
> there some value common for implementations?
>
> Best regards,
> Cezary Dziemian
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20181123/e0bb5ac7/attachment.html>;

📅 Original date posted:2018-11-16 📝 Original message: Dear ...

2023-06-09T12:52:52Z

In reply to nevent1q…y6fw
_________________________

📅 Original date posted:2018-11-16
📝 Original message:
Dear Rusty,

I am not getting this proposal (maybe I am lacking some technical basic
understandings) however I decided to ask more questions in order to
complete my onboarding process faster and hope this is fine.

My problem starts with the fact that I can't find the term "lightning probe
message" in the current BOLTs (actually the term probe only occures two
times and these seem unrelated to what you are talking about) so I am
confused what this is.
As far as I understand your proposal from a high level the payer is
supposed to create an onion package which triggers the offering of HTLCs
with some additional metadata so that the receipient of the final onion can
answer with a BOLT11 invoice. What I don't get is the fact that a payment
hash needs to be known in order to offer HTLCs.
Though I imagine you ment it differently I would not see a problem with the
payer to know the preimage in advance as he is creating the entire onion on
his behalf and sponanious without invoice anyway. However I don't get why a
returned BOLT11 invoice is needed then. I assume that my previouse
statement is wrong anyway since you don't mention anywhere how the preimage
would be send from the payer to the payee.

In general I was wondering (already during the summit) why we don't include
a connection oriented communication layer on top of the current protocol
which would allow payer and payee to communicate more efficiently about
payment and routing process and to negotiate stuff like spontaneos
payments. I see two reasons against this: 1.) more synchronous
communication makes stuff more complicated to implement and 2.) privacy
concerns.
Am I missing something here? (and sorry for splitting the topic but I
didn't want to start a new one when it actually seems to fit to this
proposal.

best Rene

On Thu, Nov 15, 2018 at 4:57 AM Rusty Russell <rusty at rustcorp.com.au> wrote:

> Hi all,
>
> I want to propose a method of having reusable BOLT11 "offers" which
> provide almost-spontaneous payments as well as not requiring generating
> a BOLT11 invoice for each potential sale.
>
> An "offer" has a `p` field of 26 bytes (128 bits assuming top two are 0)
> (which is ignored by existing nodes). The payer uses a new lightning
> probe message using the current onion format we use for HTLCs to
> retreive the complete invoice.
>
> The format of the final-hop lightning onion would contain:
>
> [whatever-marker-we-need?][128-bit-`p`-field][[type,len,data]+]
>
> We would probably define a few optional types to start:
>
> 1. quantity: for ordering multiple of an item, default 1.
> 2. delivery-address: steal from
> https://www.w3.org/TR/vcard-rdf/#Delivery_Addressing_Properties ?
> 3. signature: basically a blob so payer can prove it was them.
>
> The return lightning message would contain a new bolt11 invoice (perhaps
> we optimize some fields by copying from the bolt11 offer if they don't
> appear?), and an additional field:
>
> `m` (27) `data_length` 52. Merkle hash of fields payer provided
> in onion msg above, and the offer `p` value.
>
> The payer checks the signature is correct, `m` is correct, and uses the
> invoice to pay as normal. The bolt11 offer + fields-from-onion + bolt11
> invoice + preimage is the complete proof of payment.
>
> Refinements
> -----------
>
> We can generate alternate leaves for the merkle tree (using
> SHA256(shared-secret | leafnum)) so revealing the `m` value doesn't risk
> revealing your delivery-address for example.
>
> The return needs to list the fields it *didn't* include in the merkle
> because it didn't accept them (the merchant doesn't want to be bound to
> conditions it doesn't understand!).
>
> We could add a `k` field to the bolt11 offer to allow the final invoice
> to delegated to a separate key.
>
> The default `x` (expiry) field for an offer which does not have an
> old-style 53-byte `p` field (ie. a "pure" offer) could be infinite.
>
> We could merkelize the delivery-address too :)
>
> I've handwaved a bit over the detailed format, because there are other
> things we want to put in the onion padding, and because the return is
> similar to the "soft-error"/"partial payment ack" proposals.
>
> Results
> -------
>
> This gives us static invoicing, and a single static invoice (without an
> amount field) can thus be used to approximate "spontaneous" donations,
> while still providing proof of payment; indeed, providing
> non-transferrable proof-of-payment since the invoice now commits to the
> payer-provided signature.
>
> It also provides a platform for recurring payments: while we can do this
> with preimage-is-next-payment_hash, that requires pre-generation and
> isn't compatible with static invoices.
>
> I apologize that this wasn't fleshed out before the summit, but I
> overestimated the power of Scriptless Scripts so had mentally deferred
> this.
>
> Thanks!
> Rusty.
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20181116/573e0200/attachment-0001.html>;

📅 Original date posted:2018-11-20 📝 Original message: Hey ...

2023-06-09T12:52:44Z

In reply to nevent1q…a0rf
_________________________

📅 Original date posted:2018-11-20
📝 Original message:
Hey List,

as this base AMP proposal seems pretty small I just started to write this
up to make a PR for BOLT04 and BOLT11. While doing my write up I realize
that there are smaller things that I would want to verify / double check
and propose with you.

## Verifying:
1.) I understand the receiving node signals support for Base AMP by setting
a feature bit in the BOLT11 String
2.) The sending node signals a multipath payment by setting a feature bit
and by using the same `amount to forward` value in the last hop of the
onion for all paths which will also be bigger that the incoming htlcs whose
sum has to be at least the size of `amount_to_forward`.

## Clarifying:
3.) Senders MUST NOT (SHOULD NOT?) create paths which would have to be
merged by intermediary nodes (as we don't know - and have no means of
querying - if they support the format of the adepted onion packages for
partial paths. Also it even seems impossible since the rest of the path for
at least one partial path could not be stored in the onion / forwarded
onions can't be seen)

## Proposing:
Should we specify an algorithm for executing a multipath payment for the
sending node or should this be left to the implementation. An obvious Idea
for an algorithm would be a divide and conquer scheme which should be
obvious with the following python style pseudo code:

def pay_base_amp(amount):
success = False
for route in get_available_routes():
success = send_via_route(route, amount)
if not success:
pay_base_amp(amount/2 + 1) # the +1 is to mitigate rounding errors.
there could be other ways to do so.
pay_base_amp(amount/2 + 1)

Even if we leave the exact AMP execution to the sender we could still
suggest this divide and conquer scheme in BOLT 04

Another idea I had (which is probably a bad one as it allows for probing of
channel balances) would be to allow nodes on a partial path to send back
some hints of how much additional capacity they can forward if they see
that the partial payment feature bit is set (this would require to set this
feature bit in every onion) Also if we want to make use of this information
every node would have to support base amp. So I guess this idea is bad for
several reasons. Still we could have a MAY rule out of it?

best Rene

On Fri, Nov 16, 2018 at 4:45 PM Anthony Towns <aj at erisian.com.au> wrote:

> On Thu, Nov 15, 2018 at 11:54:22PM +0000, ZmnSCPxj via Lightning-dev wrote:
> > The improvement is in a reduction in `fee_base_msat` in the C->D path.
>
> I think reliability (and simplicity!) are the biggest things to improve
> in lightning atm. Having the flag just be incuded in invoices and not
> need to be gossiped seems simpler to me; and I think endpoint-only
> merging is better for reliability too. Eg, if you find candidate routes:
>
> A -> B -> M -- actual directed capacity $6
> A -> C -> M -- actual directed capacity $5.50
> M -> E -> F -- actual directed capacity $6
> A -> X -> F -- actual directed capacity $7
>
> and want to send $9 form A to F, you might start by trying to send
> $5 via B and $4 via C.
>
> With endpoint-only merging you'd do:
>
> $5 via A,B,M,E,F -- partial success
> $4 via A,C,M,E -- failure
> $4 via A,X,F -- payment completion
>
> whereas with in-route merging, you'd do:
>
> $5 via A,B,M -- held
> $4 via A,C,M -- to be continued
> $9 via M,E -- both partial payments fail
>
> which seems a fair bit harder to incrementally recover from.
>
> > Granted, current `fee_base_msat` across the network is very low
> currently.
> > So I do not object to restricting merge points to ultimate payees.
> > If fees rise later, we can revisit this.
>
> So, while we already agree on the approach to take, I think the above
> provides an additional rationale :)
>
> Cheers,
> aj
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20181120/af408cd6/attachment.html>;

📅 Original date posted:2018-08-27 📝 Original message: Hey ...

2023-06-09T12:51:24Z

In reply to nevent1q…6fmj
_________________________

📅 Original date posted:2018-08-27
📝 Original message:
Hey Kulpreet,

thanks for this nice overview article! I have just today implemented a
first draft for the c-lightning autopilot [0]. I have implemented 4
heuristics to select nodes to which one could connect. On of those [1]
samples from the nodes that contribute to the high diameter. This heuristic
was included not to increase the utility of the node that is running the
autopilot but to improve the network properties. I believe that this
heuristic should also reduce the articulation points and biconnected
components that you mention in your article. As endpoints of longest
shortest paths will most likely be in two different biconnectivity
components (at least if those exist and have a certain size).

Regarding the centrality. I also calculated the betweeness centrality and
have similar results [2] to yours. I guess the difference will be due to
the fact that we don't work on the exact same snapshot. My autopilot
implementation also connects to a few rather central nodes. I doubt this is
useful for the network but I guess it is good for the node running the
autopilot since it gains access to many nodes. ( Actually I think - but
don't know - that in combination with [1] it even helps the network).

Regarding your 200 Articulation points I would guess that many of those are
just nodes that only have one channel with the node that acts as an
articulation point. I guess this is not something that we would need to
take care of so much since it is also in the responsibility of those nodes
to have more than one channel. for larger biconnectivity components the
problem would probably be resolved with the above mentioned heuristic.
Therefor I believe looking at the articulation points should not be our
main focus.

Something that (regarding the autopilot) I am currently missing in your
article is how much funds should be allocated for the suggested channels. I
am currently experimenting with a probability density function that is
proportional to the average capacity of each node in the candidate set. I
smooth this with a uniform distribution. However simulations at this point
are quite expensive (if done primitively since the centralities have to be
recomputed) I guess this would be a nice future work. I will probably
tomorrow publish the rest of the code for the lib-autopilot that uses this
heuristic for channel balances since I am currently still working on it.

If you consider working more on the autopilot but also on research related
to this I would also suggest the following resources [3],[4] and [5]

[0] https://github.com/ElementsProject/lightning/pull/1888
[1]
https://github.com/renepickhardt/lightning/blob/8c91f57490b51f772513a274d679d3ab62e7201a/contrib/lib-autopilot.py#L205
[2] https://twitter.com/renepickhardt/status/1034066602273193985
[3] https://github.com/lightningnetwork/lnd/issues/677
[4]
https://github.com/renepickhardt/Automatically-Generating-a-Robust-Topology-for-the-Lightning-Network-on-top-of-Bitcoin
[5]
https://www.rene-pickhardt.de/improve-the-autopilot-of-bitcoins-lightning-network-summary-of-the-bar-camp-session-at-the-2nd-lightninghackday-in-berlin/

best regards Rene

On Mon, Aug 27, 2018 at 11:59 PM Kulpreet Singh <zapfmann at gmail.com> wrote:

> Hi all,
>
> I have been thinking about how we could measure the centrality of
> various nodes in the LN graph and the dependence on some nodes to
> route payments and to prevent network partitions. I think measuring
> and tracking the changes in key metrics could help the community
> decide on which nodes to open new channels with.
>
> I measured the centrality of nodes and the central point dominance as
> defined in the seminal paper by Lindon C. Freeman, "A Set of Measures
> of Centrality Based on Betweenness", Sociometry 40, pp. 35-41, 1977.
>
> I also measured the number of articulation points in the network as
> per Robert E. Tarjan, "Depth first search and linear graph algorithms"
> SIAM Journal on Computing, 1(2):146-160, 1972.
>
> I want to add, that this is just a start, I understand that we should
> probably look at treating LN as a directed graph and that we should do
> some work in trying to do some analysis based on treating LN as a flow
> network.
>
> However, I am eager to share my early results and would welcome any
> feedback or suggestions on the way forward.
>
> I wrote a medium post describing the approach and show my results
> there. I also elaborate on the choice of the two metrics and what they
> mean for LN. The post is available here:
>
> https://medium.com/@jungly/measuring-node-centrality-in-lightning-network-8102a59999f0
>
> Looking forward to your suggestions and feedback.
>
> Thanks
> Kulpreet
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20180828/83b36302/attachment-0001.html>;

📅 Original date posted:2018-08-23 📝 Original message: Hey ...

2023-06-09T12:51:22Z

In reply to nevent1q…ktcg
_________________________

📅 Original date posted:2018-08-23
📝 Original message:
Hey lightning devs,

I was wondering if any of the companies here are members of W3C and if
anyone here could be member of the W3C Web Payments Working Group (c.f.:
https://www.w3.org/Payments/WG/ )? According to this mail
https://lists.linuxfoundation.org/pipermail/lightning-dev/2018-March.txt
Christian Decker is a member. Which I think would be awesome!

They have just released their candidate recommendation for a payment API
at: https://www.w3.org/TR/payment-request/ According to their site the
proposed recommendation will be published not earlier than October 31st
2018. They are currently looking for feedback in their github repository
at: https://github.com/w3c/payment-request/

I can see that they have bitcoin somewhat on their mind. But I guess it
would be even cooler if we could make sure that lightning payments will
also be compatible with their recommendation.

Christian - if you really are a member - could you give us an update on
that work? How relevant is it for us?

best Rene

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20180823/b8855da4/attachment.html>;

📅 Original date posted:2018-07-22 📝 Original message: Hey ...

2023-06-09T12:51:10Z

In reply to nevent1q…wqev
_________________________

📅 Original date posted:2018-07-22
📝 Original message:
Hey everyone,

in the grand tradition of BIPs I propose that we also start to have our own
LIPs (Lightning Network Improvement proposals)

I think they should be placed on the github.com/lightning account in a repo
called lips (or within the lightning rfc repo) until that will happen I
created a draft for LIP-0001 (which is describing the process and is 95%
influenced by BIP-0002) in my github repo:

https://github.com/renepickhardt/lips (There are some open Todos and
Questions in this LIP)

The background for this Idea: I just came home from the bitcoin munich
meetup where I held a talk examining BOLT. As I was asked to also talk
about the future plans of the developers for BOLT 1.1 I realized while
preparing the talk that many ideas are distributed within the community but
it seems we don't have a central place where we collect future enhancements
for BOLT1.1. Having this in mind I think also for the meeting in Australia
it would be nice if already a list of LIPs would be in place so that the
discussion can be more focused.
potential LIPs could include:
* Watchtowers
* Autopilot
* AMP
* Splicing
* Routing Protcols
* Broadcasting past Routing statistics
* eltoo
* ...

As said before I would volunteer to work on a LIP for Splicing (actually I
already started)

best Rene

--
https://www.rene-pickhardt.de

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20180722/a4d1ce8b/attachment.html>;

📅 Original date posted:2018-07-01 📝 Original message: Hey ...

2023-06-09T12:50:59Z

In reply to nevent1q…jam0
_________________________

📅 Original date posted:2018-07-01
📝 Original message:
Hey Dmytro,

thank your for your solid input to the discussion. I think we need to
consider that the setting in the lightning network is not exactly
comparable to the one described in the 2010 paper.

1st: the paper states in section 5.2: "It appears that a mathematical
analysis of a transaction routing model where intermediate nodes charged a
routing fee would require an entirely new approach since it would
invalidate the cycle-reachability relation that forms the basis of our
results."
Since we have routing fees in the lightning network to my understanding the
theorem and lemma you quoted in your medium post won't hold.

2nd: Even if we neglect the routing fees and assume the theorem still holds
true we have two conditions that make the problem way more dynamic:
A) In the lightning network we do not know the weights of the directed
edges. (only the sum of two opposing edges) So while theoretically the flow
in the network will only depend on the liquidity of the nodes I guess in
practice well balanced channels will increase the probability to actually
find a working route.
B) I believe the HTLCs create a situation where funds are being locked up
while routing takes place and thus have an impact to the entire flow of the
network. While Alice searches for a route for her payment a proper route
could be blocked do to the fact that Bob is using it currently. Since the
funds of Bob have not arrived Alice might also not be able to find a
different route.

However those scientific results are a strong call for Atomic Multipath
Payments. I personally think they are also a strong call for splicing since
this allows to easilly increase the flow of the network by updating a
channel (athough you might argue that following the paper this could be
achieved by just creating a new channel)

best Rene

On Sun, Jul 1, 2018 at 12:21 PM Dmytro Piatkivskyi <
dmytro.piatkivskyi at ntnu.no> wrote:

> Hi everyone,
>
> I have been working academically on the Lightning network for a while now.
> I didn’t not participate in the list to form my own vision of what it
> should be. So please, bear with me if I’ll be saying nonsense sometimes.
>
> There has been a lot of discussion on sending cycle transactions to
> oneself to ‘re-balance’ the network. On LN mailing list
> <https://lists.linuxfoundation.org/pipermail/lightning-dev/2018-February/001005.html>; [1] or
> numerous places elsewhere. There has been even a paper suggesting a smart
> mechanism to do the re-balancing (see Revive or Liquidity network [2]). My
> question is what do we actually get from it? [3] states that the
> distribution of funds in channels does not really affect the network
> liquidity. I can see cheaper fees or shorter paths if the network is kept
> balanced. But don’t you think that a smart fee strategy will do the job?
>
> To save your time, [4] explains the gist from [3].
>
> [1]
> https://lists.linuxfoundation.org/pipermail/lightning-dev/2018-February/001005.html
> [2]
> https://www.reddit.com/r/ethereum/comments/7bse33/were_very_happy_to_announce_the_liquiditynetwork/
> [3] https://arxiv.org/abs/1007.0515
> [4]
> https://medium.com/@dimapiatkivskyi/why-would-you-re-balance-a-payment-network-796756ad4f31
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

--
www.rene-pickhardt.de
<http://www.beijing-china-blog.com/>;

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20180701/288b5422/attachment.html>;

📅 Original date posted:2018-03-13 📝 Original message: Hey ...

2023-06-09T12:49:29Z

In reply to nevent1q…4ea5
_________________________

📅 Original date posted:2018-03-13
📝 Original message:
Hey Christian,

I agree with you on almost anything you said. however I disagree that in
the lightning case it produces just another double spending. I wish to to
emphasize on my statement that the in the case with lightning such a 51%
attack can steal way more BTC than double spending my own funds. The
following example is a little extrem and constructed but it should help to
make the point. Also for pure convenience reasons I neglected the fact that
channels should never be worse distributed than 99% to 1%:

Let us assume I am the attacker currently owning 1000 BTC. Now 1000 nodes
called n_0,...n_{999} open a payment channel with me (all funded by the
other side with 999 BTC in each channel (and 1 BTC from me)) resulting in
the following channel balance sheet:
c_0: me = 1 BTC and n_0 = 999 BTC
c_1: me = 1 BTC and n_1 = 999 BTC
c_2: me = 1 BTC and n_2 = 999 BTC
...
c_{999}: me = 1 BTC and n_{999} = 999 BTC

Now node n_0 sends 1 BTC to each node n_1,...,n_{999} (using me for routing
the payment) so the channel balances read:
c_0: me = 1000 BTC and n_0 = 0 BTC (save the corresponding
commitment transaction!)
c_1: me = 0 BTC and n_1 = 1000 BTC
c_2: me = 0 BTC and n_2 = 1000 BTC
...
c_{999}: me= 0 BTC and n_{999} = 1000 BTC

next n_1 sends 1000 BTC to n_0:
c_0: me = 0 BTC and n_0 = 1000 BTC
c_1: me = 1000 BTC and n_1 = 0 BTC (save the corresponding
commitment transaction!)
c_2: me = 0 BTC and n_2 = 1000 BTC
...
c_{999}: me= 0 BTC and n_{999} = 1000 BTC

similarly n_2 sends 1000 BTC to n_1:
c_0: me = 0 BTC and n_0 = 1000 BTC
c_1: me = 0 BTC and n_1 = 1000 BTC
c_2: me = 1000 BTC and n_2 = 0 BTC (save the corresponding
commitment transaction!)
...
c_{999}: me = 0 BTC nad n_{999} = 1000 BTC

following this scheme n_3 --[1000 BTC]--> n_2, n_4 --[1000 BTC]--> n_3,...

due to this (as mentioned highly constructed and artificial behavior) I
will have old commitment transactions in *each* and every channel (which
spends 1000 BTC to me)

When starting my secret mining endeavor I spend those commitment
transactions which gives in this particular case 1000 * 1000 BTC = 1M BTC
to me.

So while I agree that a 51% is a problem for any blockchain technology I
think the consequences in the lightning scenario are way more problematic
and makes such an attack also way more interesting for a dishonest
fraudulent person / group. In particular I could run for a decade on stable
payment channels storing old state and at some point realizing it would be
a really big opportunity secretly cashing in all those old transactions
which can't be revoked.

I guess one way of resolving this kind of limitless but rare possibility
for stealing could be to make sure no one can have more than 2 or three
times the amount of BTC she owns in all the payment channels the person has
open. As funding transactions are publicly visible on the blockchain one
could at least use that measure to warn people before opening and funding
another payment channel with a node that is heavily underfunded. Also in
the sense of network topology such a measure would probably make sure that
channels are somewhat equally funded.

best Rene

On Tue, Mar 13, 2018 at 3:55 PM, Christian Decker <
decker.christian at gmail.com> wrote:

> Hi René,
>
> very good question. I think the simple answer is that this is exactly
> the reason why not having a participant in the network that can 51%
> attack over a prolonged period is one of the base assumptions in
> Lightning. These attacks are deadly to all blockchains, and we are
> certainly no different in that regard.
>
> More interesting is the assertion that this may indeed be more dangerous
> than a classical 51% attack, in which an attacker can only doublespend
> funds that she had control over at some point during the attack
> (duration being defined as the period she can build a hidden fork of). I
> think the case for Lightning is not more dangerous since what they could
> do is enforce an old state in which they had a higher balance than in
> the final state, without incurring in a penalty. The key observation is
> that in this old state they actually had to have the balance they are
> stealing on the channel. So this maps directly to the classical
> scenario in which an attacker simply doublespends funds they had control
> over during the attack, making the attack pretty much the same.
>
> Another interesting observation is that with Lightning the state that
> the attacker is enforcing may predate the attack, e.g., an attacker
> could use a state that existed and was replaced before it started
> generating its fork. This is in contrast to the classical doublespend
> attack in which invalidated spends have to happen after the fork
> started, and the attacker just filters them from its fork.
>
> But as I said before, if we can't count on there not being a 51%
> attacker, then things are pretty much broken anyway :-)
>
> Cheers,
> Christian
>
> René Pickhardt via Lightning-dev
> <lightning-dev at lists.linuxfoundation.org> writes:
> > Hey everyone,
> >
> > disclaimer: as mentioned in my other mail (
> > https://lists.linuxfoundation.org/pipermail/lightning-dev/
> 2018-March/001065.html
> > ) I am currently studying the revocation system of duplex micropayment
> > channels in detail but I am also pretty new to the topic. So I hope the
> > attack I am about to describe is not possible and it is just me
> overseeing
> > some detail or rather my lack of understanding.
> > That being said even after waiting one week upon discovery and double
> > checking the assumptions I made I am still positive that the revocation
> > system in its current form allows for a new form of a 51% attack. This
> > attack seems to be way more harmful than a successful 51% attack on the
> > bitcoin network. Afaik within the bitcoin network I could 'only double
> > spend' my own funds with a successful 51% attack. In the lightning case
> it
> > seems that an attacker could steal an arbitrary amount of funds as long
> as
> > the attacker has enough payment channels with enough balance open.
> >
> > The attack itself follows exactly the philosophy of lightning: "If a tree
> > falls in the forest and no one is around to hear it. Does it make a
> sound?"
> > In the context of the attack this would translate to: "If a 51% attacker
> > secretly mines enough blocks after fraudulently spending old commitment
> > transactions and no one sees it during the the *to_self_delay* period,
> > have the commitment transactions been spent? (How) Can they be revoked?"
> >
> >
> > As for the technical details I quote from the spec of BOLT 3:
> > "*To allow an opportunity for penalty transactions, in case of a revoked
> > commitment transaction, all outputs that return funds to the owner of the
> > commitment transaction (a.k.a. the "local node") must be delayed for *
> > *to_self_delay** blocks. This delay is done in a second-stage HTLC
> > transaction (HTLC-success for HTLCs accepted by the local node,
> > HTLC-timeout for HTLCs offered by the local node)*"
> >
> > Assume an attacker has 51% of the hash power she could open several
> > lightning channels and in particular accept any incoming payment channel
> > (the more balance is in her channels the more lucrative the 51% attack).
> > Since the attacker already has a lot of hash power it is reasonable (but
> > not necessary) to assume that the attacker already has a lot of bitcoins
> > and is well known to honest nodes in the network which makes it even more
> > likely to have many open channels.
> >
> > The attacker keeps track of her (revocable) commitment transactions in
> > which the balance is mostly on the attackers side. Once the attacker
> knows
> > enough of these (old) commitment transactions the attack is being
> executed
> > in the following way:
> > 0.) The max value of to_self_delay is evaluated. Let us assume it is 72
> > blocks (or half a day).
> > 1.) The attacker secretly starts mining on her own but does not
> broadcasts
> > any successfully mined block. Since the attacker has 51% of the hash
> power
> > she will most likely be faster than the network to mine the 72 blocks of
> > the safety period in which fraudulent commitment transactions could be
> > revoked.
> > 2.) The attacker spends all the fraudulent (old) commitment transactions
> in
> > the first block of her secrete mining endeavor.
> > 3.) Meanwhile the attacker starts spending her own funds of her payment
> > channels e.g on decentralized exchanges for any other (crypto)currency.
> > 4.) As soon as the attacker has mined enough blocks that the commitment
> > transactions cannot be revoked she broadcasts her secretly minded
> > blockchain which will be accepted by the network as it is the longest
> > chain. (In Particular she includes all the other bitcoin transactions
> that
> > are also in the original public blockchain so that other people don't
> even
> > realize something suspicious has happened.)
> >
> > Since according to the spec channels should never be balanced worse than
> > 99% to 1% the attacker could steal up to 99% of all the bitcoins
> allocated
> > in the sum of all payment channels the attacker was connected to. This
> > amount could obviously be way higher than just double spending her own
> > funds. This attack would be interesting in particular for the power nodes
> > created by the Barabasi-Albert model of lnd's autopilot (c.f.:
> > https://github.com/lightningnetwork/lnd/issues/677 ).
> >
> > I understand that with the growth of the bitcoin (mining) network a 51%
> > attack becomes less and less likely. Also I am very happy to be proven
> > false about the attack that I am describing.
> >
> > Another sad thing about this attack is that I currently do not see any
> > (reasonable) way of preventing this form of a 51% attack (other than
> > creating payment channels that don't offer the possibility of revocation)
> > as it is abusing exactly the core idea of lightning to do something in
> > secret without broadcasting it.
> >
> > Best regards Rene
> >
> > ---
> >
> > http://www.rene-pickhardt.de
> > _______________________________________________
> > Lightning-dev mailing list
> > Lightning-dev at lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>

--
www.rene-pickhardt.de
<http://www.beijing-china-blog.com/>;

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20180313/b1e24dd5/attachment-0001.html>;

📅 Original date posted:2018-03-13 📝 Original message: Hey ...

2023-06-09T12:49:29Z

In reply to nevent1q…9kd3
_________________________

📅 Original date posted:2018-03-13
📝 Original message:
Hey everyone,

disclaimer: as mentioned in my other mail (
https://lists.linuxfoundation.org/pipermail/lightning-dev/2018-March/001065.html
) I am currently studying the revocation system of duplex micropayment
channels in detail but I am also pretty new to the topic. So I hope the
attack I am about to describe is not possible and it is just me overseeing
some detail or rather my lack of understanding.
That being said even after waiting one week upon discovery and double
checking the assumptions I made I am still positive that the revocation
system in its current form allows for a new form of a 51% attack. This
attack seems to be way more harmful than a successful 51% attack on the
bitcoin network. Afaik within the bitcoin network I could 'only double
spend' my own funds with a successful 51% attack. In the lightning case it
seems that an attacker could steal an arbitrary amount of funds as long as
the attacker has enough payment channels with enough balance open.

The attack itself follows exactly the philosophy of lightning: "If a tree
falls in the forest and no one is around to hear it. Does it make a sound?"
In the context of the attack this would translate to: "If a 51% attacker
secretly mines enough blocks after fraudulently spending old commitment
transactions and no one sees it during the the *to_self_delay* period,
have the commitment transactions been spent? (How) Can they be revoked?"

As for the technical details I quote from the spec of BOLT 3:
"*To allow an opportunity for penalty transactions, in case of a revoked
commitment transaction, all outputs that return funds to the owner of the
commitment transaction (a.k.a. the "local node") must be delayed for *
*to_self_delay** blocks. This delay is done in a second-stage HTLC
transaction (HTLC-success for HTLCs accepted by the local node,
HTLC-timeout for HTLCs offered by the local node)*"

Assume an attacker has 51% of the hash power she could open several
lightning channels and in particular accept any incoming payment channel
(the more balance is in her channels the more lucrative the 51% attack).
Since the attacker already has a lot of hash power it is reasonable (but
not necessary) to assume that the attacker already has a lot of bitcoins
and is well known to honest nodes in the network which makes it even more
likely to have many open channels.

The attacker keeps track of her (revocable) commitment transactions in
which the balance is mostly on the attackers side. Once the attacker knows
enough of these (old) commitment transactions the attack is being executed
in the following way:
0.) The max value of to_self_delay is evaluated. Let us assume it is 72
blocks (or half a day).
1.) The attacker secretly starts mining on her own but does not broadcasts
any successfully mined block. Since the attacker has 51% of the hash power
she will most likely be faster than the network to mine the 72 blocks of
the safety period in which fraudulent commitment transactions could be
revoked.
2.) The attacker spends all the fraudulent (old) commitment transactions in
the first block of her secrete mining endeavor.
3.) Meanwhile the attacker starts spending her own funds of her payment
channels e.g on decentralized exchanges for any other (crypto)currency.
4.) As soon as the attacker has mined enough blocks that the commitment
transactions cannot be revoked she broadcasts her secretly minded
blockchain which will be accepted by the network as it is the longest
chain. (In Particular she includes all the other bitcoin transactions that
are also in the original public blockchain so that other people don't even
realize something suspicious has happened.)

Since according to the spec channels should never be balanced worse than
99% to 1% the attacker could steal up to 99% of all the bitcoins allocated
in the sum of all payment channels the attacker was connected to. This
amount could obviously be way higher than just double spending her own
funds. This attack would be interesting in particular for the power nodes
created by the Barabasi-Albert model of lnd's autopilot (c.f.:
https://github.com/lightningnetwork/lnd/issues/677 ).

I understand that with the growth of the bitcoin (mining) network a 51%
attack becomes less and less likely. Also I am very happy to be proven
false about the attack that I am describing.

Another sad thing about this attack is that I currently do not see any
(reasonable) way of preventing this form of a 51% attack (other than
creating payment channels that don't offer the possibility of revocation)
as it is abusing exactly the core idea of lightning to do something in
secret without broadcasting it.

Best regards Rene

---

http://www.rene-pickhardt.de
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20180313/bb92bcf7/attachment-0001.html>;

📅 Original date posted:2018-03-01 📝 Original message: Hey ...

2023-06-09T12:49:18Z

In reply to nevent1q…advd
_________________________

📅 Original date posted:2018-03-01
📝 Original message:
Hey everyone,

disclaimer I am new here and have not a full understanding of the complete
specs yet - however since I decided to participate in lighting dev I will
just be brave and try to add my ideas on the problem jimpo posed. So even
in case by ideas are complete bs please just tell me in a friendly way and
I know I need to read more code and specs in order to participate.

Reading about the described problem techniques like IP-Fragmentation (
https://en.wikipedia.org/wiki/IP_fragmentation ) come to my mind. The
setting is a little bit different but in from my current understanding it
should still be applicable and also be the favorable solution in comparison
to the proposed ping:

1.) IP setting: In IP-Fragmentation one would obviously just split the
IP-package in order to utilize a link layer protocol that doesn't have
enough bandwidth for a bigger size package.
2.) Lightning case: When the capacity of a channel during routing is not
high enough - which means that the channel balance on that side is
somewhere between 0 and the size of the payment - one would have to to send
the second part of the fragmented package on a different route. This is
obvious since the insufficient channel balance cannot come out of thin air
(as in IP-Routing).

My first intuition was that this would become a problem for onion routing
since the router in question does not know the final destination but only
knows the next hop which can't be utilized as the channel doesn't have
enough funds. However I imagine one could just encapsulate the second part
of the fragmented payment in a new onion routed package that goes on a
detour (using different payment channels) to the original "next" hop und
progresses from there as it was originally thought of.

Not sure however how the impacts to the HTLC would be and if it would
actually be possible to fragment a payment that is encapsulated within the
onion routing.

If possible the advantage in comparison to your proposed ping method is
that fragmentation would be highly dynamic and still work if a channel runs
out of funds while routing payment. In your ping scenario it could very
well happen that you do a ping for a designated rout, everything looks
great, you send a payment but while it is on its way a channel on that way
could run dry.

best Rene

On Thu, Mar 1, 2018 at 3:45 PM, Jim Posen <jim.posen at gmail.com> wrote:

> My understanding is that the best strategy for choosing a route to send
> funds over is to determine all possible routes, rank them by estimated fees
> based on channel announcements and number of hops, then try them
> successively until one works.
>
> It seems inefficient to me to actually do a full HTLC commitment handshake
> on each hop just to find out that the last hop in the route didn't have
> sufficient remaining capacity in the first place. Depending on how many
> people are using the network, I could also forsee situations where this
> creates more payment failures because bandwidth is locked up in HTLCs that
> are about to fail anyway.
>
> One idea that would likely help is the ability to send a ping over an
> onion route asking "does every hop have capacity to send X msat?" Every hop
> would forward the onion request if the answer is yes, or immediately send
> the response back up the circuit if the answer is no. This should reveal no
> additional information about the channel capacities that the sender
> couldn't determine by sending a test payment to themself (assuming they
> could find a loop). Additionally, the hops could respond with the latest
> fee rate in case channel updates are slow to propagate.
>
> The main benefit is that this should make it quicker to send a successful
> payment because latency is lower than sending an actual payment and the
> sender could ping all possible routes in parallel, whereas they can't send
> multiple payments in parallel. The main downside I can think of is that,
> by the same token, it is faster and cheaper for someone to extract
> information about channel capacities on the network with a binary search.
>
> -jimpo
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
>

--
Skype: rene.pickhardt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20180301/ba0d1507/attachment.html>;

📅 Original date posted:2018-03-01 📝 Original message: Hey ...

2023-06-09T12:49:17Z

In reply to nevent1q…l2gd
_________________________

📅 Original date posted:2018-03-01
📝 Original message:
Hey everyone,

disclaimer I am new here and have not a full understanding of the complete
specs yet - however since I decided to participate in lighting dev I will
just be brave and try to add my ideas on the problem jimpo posed. So even
in case by ideas are complete bs please just tell me in a friendly way and
I know I need to read more code and specs in order to participate.

Reading about the described problem techniques like IP-Fragmentation (
https://en.wikipedia.org/wiki/IP_fragmentation ) come to my mind. The
setting is a little bit different but in from my current understanding it
should still be applicable and also be the favorable solution in comparison
to the proposed ping:

1.) IP setting: In IP-Fragmentation one would obviously just split the
IP-package in order to utilize a link layer protocol that doesn't have
enough bandwidth for a bigger size package.
2.) Lightning case: When the capacity of a channel during routing is not
high enough - which means that the channel balance on that side is
somewhere between 0 and the size of the payment - one would have to to send
the second part of the fragmented package on a different route. This is
obvious since the insufficient channel balance cannot come out of thin air
(as in IP-Routing).

My first intuition was that this would become a problem for onion routing
since the router in question does not know the final destination but only
knows the next hop which can't be utilized as the channel doesn't have
enough funds. However I imagine one could just encapsulate the second part
of the fragmented payment in a new onion routed package that goes on a
detour (using different payment channels) to the original "next" hop und
progresses from there as it was originally thought of.

Not sure however how the impacts to the HTLC would be and if it would
actually be possible to fragment a payment that is encapsulated within the
onion routing.

If possible the advantage in comparison to your proposed ping method is
that fragmentation would be highly dynamic and still work if a channel runs
out of funds while routing payment. In your ping scenario it could very
well happen that you do a ping for a designated rout, everything looks
great, you send a payment but while it is on its way a channel on that way
could run dry.

best Rene

On Thu, Mar 1, 2018 at 3:45 PM, Jim Posen <jim.posen at gmail.com> wrote:

> My understanding is that the best strategy for choosing a route to send
> funds over is to determine all possible routes, rank them by estimated fees
> based on channel announcements and number of hops, then try them
> successively until one works.
>
> It seems inefficient to me to actually do a full HTLC commitment handshake
> on each hop just to find out that the last hop in the route didn't have
> sufficient remaining capacity in the first place. Depending on how many
> people are using the network, I could also forsee situations where this
> creates more payment failures because bandwidth is locked up in HTLCs that
> are about to fail anyway.
>
> One idea that would likely help is the ability to send a ping over an
> onion route asking "does every hop have capacity to send X msat?" Every hop
> would forward the onion request if the answer is yes, or immediately send
> the response back up the circuit if the answer is no. This should reveal no
> additional information about the channel capacities that the sender
> couldn't determine by sending a test payment to themself (assuming they
> could find a loop). Additionally, the hops could respond with the latest
> fee rate in case channel updates are slow to propagate.
>
> The main benefit is that this should make it quicker to send a successful
> payment because latency is lower than sending an actual payment and the
> sender could ping all possible routes in parallel, whereas they can't send
> multiple payments in parallel. The main downside I can think of is that,
> by the same token, it is faster and cheaper for someone to extract
> information about channel capacities on the network with a binary search.
>
> -jimpo
>
> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
>

--
www.rene-pickhardt.de
<http://www.beijing-china-blog.com/>;

Skype: rene.pickhardt

mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20180301/de74d39d/attachment.html>;

📅 Original date posted:2022-01-11 📝 Original message:jack ...

2023-06-07T23:02:10Z

In reply to nevent1q…shc6
_________________________

📅 Original date posted:2022-01-11
📝 Original message:jack via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org> schrieb am
Mi., 12. Jan. 2022, 01:35:

> To Bitcoin Developers:
>
> Open-source developers, who are often independent, are especially
> susceptible to legal pressure.

Will the fund eventually also help to educate developers about the risks
they are facing and which measures can be taken to reduce such risks so
that legal pressure might not even arise in the first place?

The main purpose of this Fund is to defend developers from lawsuits
> regarding their activities in the Bitcoin ecosystem, including finding and
> retaining defense counsel, developing litigation strategy, and paying legal
> bills. This is a free and voluntary option for developers to take advantage
> of if they so wish.

Thank you!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20220112/f8dc7cf8/attachment.html>;

📅 Original date posted:2021-05-15 📝 Original message:Hey ...

2023-06-07T22:53:24Z

In reply to nevent1q…9fvr
_________________________

📅 Original date posted:2021-05-15
📝 Original message:Hey Michael,

First I think the idea of "do nothing in the first 9 minutes" will
unfortunately not be useful as the computed work is mainly there to prevent
miners from altering the history of previous blocks. Thus following your
suggesting would probably drastically decease the security of the network
as less work protects previously mined blocks allowing for lower cost to
compute a reorg.

Let us assume the aforementioned point could somehow be resolved there
would be another practical issue. It is hard / impossible to sync clocks in
a distributed network which I think would be necessary for a system that
you propose. (actually Bitcoin is one way of introducing a temporal
ordering of events in a distributed network)

Finally even if that was also resolved: How would you prevent miners to
already compute the simpler difficulty problem directly after the block was
found and publish their solution directly after minute 9? We would always
have many people with a finished / competing solution.

So while I appreciate the idea I have the feeling it would be impractical
but maybe I missed something.

Best Rene

Michael Fuhrmann via bitcoin-dev <bitcoin-dev at lists.linuxfoundation.org>
schrieb am Sa., 15. Mai 2021, 23:57:

> Hello,
>
> Bitcoin should create blocks every 10 minutes in average. So why do
> miners need to mine the 9 minutes after the last block was found? It's
> not necessary.
>
> Problem: How to prevent "pre-mining" in the 9 minutes time window?
>
> Possible ideas for discussion:
>
> - (maybe most difficult) global network timer sending a salted hash time
> code after 9 minutes. this enables validation by nodes.
>
> - (easy attempt) mining jobs before 9 minutes have a 10 (or 100 or just
> high enough) times higher difficulty. so everyone can mine any time but
> before to 9 minutes are up there will be a too high downside. It is more
> efficient to wait then paying high bills. The bitcoin will get a "puls".
>
>
> I dont think I see all problems behind these ideas but if there is a
> working solution to do so then the energy fud will find it's end. Saving
> energy without loosing rosbustness.
>
>
>
> :)
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/attachments/20210516/eac0b836/attachment.html>;