René Pickhardt [ARCHIVE] on Nostr: 📅 Original date posted:2019-03-14 📝 Original message: Hey everyone, I am glad ...
📅 Original date posted:2019-03-14
📝 Original message:
Hey everyone,
I am glad the suggestion is being picked up. At this time I want to respond
to two of the concerns that have been thrown in. I have some other comments
and ideas but would like to hold them back so that we can have more people
joining the discussion without bias also this mail will already be quite
long.
ZmnSCPxj suggested to introduce a `success_rate` for JIT routing. While
this success_rate can obviously only be estimated or configured I would
advice against including this to the protocol. As I mentioned before I
suggested to include JIT Routing as a MAY Recommendation so it is up to the
node to decide if it cannot earn `offered_fee_amount` to engage in the
JIT-rebalancing operation. A node operator might be willing in general to
to pay a fee for rebalancing even if there is not an outstanding routing
event taking place. So even while `rebalancing_fee_amount` >
`offered_fee_amount` the node could see the offered_fee_amount as a
discount for the planned rebalancing amount. We don't know that and I
honestly believe that the protocol should not make economical decisions for
the node. In any case rebalancing will overall increase the likelihood for
successful routing but it makes sense to defer the rebalancing operation to
a moment in which the liquidity is actually needed.
Regarding Ariels suggestion about reusing the payment hash with JIT Routing
I have some more thoughts:
Reusing the payment hash indeed seems like a good idea. However it produces
some technical issues which in my opinion can all be mitigated. it is just
a question with these challenges if it is worthwhile doing it?
I have drawn several situations and tried to construct an example in which
using the same payment hash for the JIT-rebalancing would result in a
severe problem with the payment process in the sense that it would be
compromised or somebody could steal funds. It could however be a privacy
issue as more nodes are being aware of the same payment (but that is also
the case with base-AMP)
I was not able to construct such a situation (under the assumption that the
rebalancing amount does not exceed the original payment amount). My feeling
(though I have not done this yet) is that one should be able to proof that
taking the same payment hash would always work and in fact create a
situation in which at least the rebalancing only takes place if the entire
payment was routed successfully.
Assuming someone will be able to proof that using the same payment hash for
JIT Routing is not an issue we still run into another problem (which I
believe can be fixed with another MUST rule but comes with quite some
implementation overhead.)
The deadlock problem when doing JIT Routing with the same payment hash:
When using the same payment hash there will be two htlc's (potentially of
different amounts) in opposing directions on the same channel (or in the
lnd case maybe between separate channels between the same two peers).
Unluckily without a novel rule this can produce a deadlock.
As an example take the situation from my initial email with an additional
recipient R1:
100 / 110 80 / 200 150/180
S ----------> B --------> R ----------> R1
\ /
80/200 \ / 100/200
\ /
T
Meaning we have the following channels:
S ---> B capacity: 110 A funds: 100 B funds: 10
B ---> R capacity: 200 B funds: 80 R funds: 120
B ---> T capacity: 200 B funds: 80 T funds: 120
T ---> R capacity: 200 T funds: 100 R funds: 100
R ---> R1 capacity: 180 R funds: 150 R1 funds: 30
neglecting fees the following htlcs would be offered
1.) S-->B amount: 90
2.) B-->T amount:50
3.) T-->R amount:50
4.) R-->B amount: 50
5.) B-->R amount: 90 (difficult to set up before 4. settles)
6.) R-->R1 amount: 90
while 1,5 and 6 are the original path 2,3,4 are the JIT rebalancing.
We see that in this situation using the same preimage results in a problem.
Since the rebalancing is not settled R will not accept the 5th htlc (B--->R
amount: 90) as there is not enough liquidity on B's side producing a
deadlock
However since the same payment hash is used it is save to combine the 4th
and 5th htlc to have the following situation:
1.) S-->B amount: 90
2.) B-->T amount:50
3.) T-->R amount:50
4.) R-->B will be removed or settled or replaced by the 5th htlcs with a
different amount (90 - 50)
5.) B-->R amount: 40
6.) R-->R1 amount: 90
Note that while theoretically it seems tempting to just have two htlc
outputs as the second node could always claim the htlc if the first claims
theirs. However this will not work onchain as potentially more funds are
spend than exist.
Therefor we need a MUST-rule to fix the deadlock problem (which could
probably also be formulated in a symmetric way):
If a node N offers an htlc to a partner with an amount x from whom the node
already received an htlc y (where y is smaller than x) the nodes must
create a new channel state discarding the old htlc but having a new offer
from N with the amount x-y.
This decreases the liquidity bound in the routing process, enables for a
critical channel to be rebalanced several times during several JIT
operations and keeps the onchain footprint low as there are less htlc
outputs.
Also as mentioned above it seems crucial that the rebalancing amount must
not exceed the original payment amount if the payment hash is being reused.
Imagine there was no R1 and R was actually the final destination and B
decides to rebalance an amount larger than necessary (which could not
happen in our setup). R could release the preimage before setting up the
final htlc from R back to B to fulfill the rebalancing request. This could
also happen if T was the final recipient (which R would not now!)
The only way how I see that this problem can be mitigated is by introducing
the following rule (morally even as a MUST rule)
If a node decides to engage in JIT Routing using the same payment hash as
the incoming htlc it SHOULD NOT rebalance an amount higher than the
incoming HTLCs. If it rebalances with a new payment hash it MAY use an
arbitrary amount.
Rational: (as described above)
Another problem while reusing the payment hash is that in this situation
the node who issued the invoice could be involved in a rebalancing act and
decline an htlc as the amount is not sufficient for the invoice. Therefor
we would have to set either the base-AMP feature flag or create a new one
for JIT-routing which would signal that there are more htlc's coming which
need to be combined to forward an onion.
In order to avoid this complex aggregation of htlcs we could also see this
process as a local AMP right a way saying that a node instead of forwarding
the onion MAY do a local base-AMP to the next recipient.
So in my opinion this suggestions to reuse the payment hash is not only
reasonable but actually desirable in particular if we can proof that it is
not a problem and if we agree that we can mitigate the technical challenges
which I described in this mail.
Best regards Rene
On Thu, Mar 14, 2019 at 2:08 PM Ariel Lorenzo-Luaces <arielluaces at gmail.com>
wrote:
> Hello Rene, ZmnSCPxj, and list
>
>
> I really like the proposal and I'm sure it's the correct way forward for
> reducing payment failures and increasing privacy (through mitigating
> probing based network analysis)
>
>
> However I am concerned that this proposal could introduce a vulnerability
> to a spoofed-payment attack.
>
>
> An adversary could create a malicious payment that's guaranteed to fail,
> for free. Any intermediary nodes on the path of the payment before it fails
> could lose fees due to rebalancing if the rebalancing path's success is not
> dependent on the original payment's success.
>
>
> Could anyone speak to this and confirm whether it would be possible for
> the rebalancing step to reuse the original payment hash? If this is
> possible then it should explicitly be included in this proposal.
>
>
> If reusing the payment hash is not possible on the routing path then JIT
> routing would need some other mitigation for the spoofed-payment attack.
>
>
> Cheers
>
>
> Ariel Lorenzo-Luaces
>
>
> On Mar 14, 2019, at 7:45 AM, ZmnSCPxj via Lightning-dev <
> lightning-dev at lists.linuxfoundation.org> wrote:
>>
>> Good morning Rene and list,
>>
>> Let us consider then the rule *when* a rebalancing would be beneficial to the node.
>>
>> The node is offered a fee amount (`offered_fee_amount`) for the forwarding.
>> It knows that, under current channel states, it will definitely have to fail and earn 0 fees.
>>
>> If it engages in JIT-routing, it must pay some fee (`rebalancing_fee_amount`) for the rebalancing operation.
>> But even if it successfully forwards its hop, it is still possible that the route will fail anyway and it will earn 0 fees.
>>
>> So let us consider the probability of success (`success_rate`), a value between 0 to 1.0.
>> This is the estimated probability that we will succeed the route after we forward it.
>>
>> We should only engage in JIT-routing if:
>>
>> offered_fee_amount * success_rate - rebalancing_fee_amount > 0
>>
>> The LHS of the subtraction is the expected earning, while the RHS of the subtraction is the expected cost.
>> The above is trivial accounting for determining net earnings.
>>
>> The fee amounts are trivially computable.
>> Presumably the rebalancing code can compute the fee given a particular rebalance path, and thus can provide `rebalancing_fee_amount`.
>>
>> The `success_rate` can be computed statically from some node data.
>> Better, is for the node to start with this precomputed static information, but augment this over time with its own experienced `success_rate` for all forwards.
>>
>> Regards,
>> ZmnSCPxj
>> ------------------------------
>>
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>>
>> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
--
https://www.rene-pickhardt.de
Skype: rene.pickhardt
mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190314/35731641/attachment-0001.html>;
📝 Original message:
Hey everyone,
I am glad the suggestion is being picked up. At this time I want to respond
to two of the concerns that have been thrown in. I have some other comments
and ideas but would like to hold them back so that we can have more people
joining the discussion without bias also this mail will already be quite
long.
ZmnSCPxj suggested to introduce a `success_rate` for JIT routing. While
this success_rate can obviously only be estimated or configured I would
advice against including this to the protocol. As I mentioned before I
suggested to include JIT Routing as a MAY Recommendation so it is up to the
node to decide if it cannot earn `offered_fee_amount` to engage in the
JIT-rebalancing operation. A node operator might be willing in general to
to pay a fee for rebalancing even if there is not an outstanding routing
event taking place. So even while `rebalancing_fee_amount` >
`offered_fee_amount` the node could see the offered_fee_amount as a
discount for the planned rebalancing amount. We don't know that and I
honestly believe that the protocol should not make economical decisions for
the node. In any case rebalancing will overall increase the likelihood for
successful routing but it makes sense to defer the rebalancing operation to
a moment in which the liquidity is actually needed.
Regarding Ariels suggestion about reusing the payment hash with JIT Routing
I have some more thoughts:
Reusing the payment hash indeed seems like a good idea. However it produces
some technical issues which in my opinion can all be mitigated. it is just
a question with these challenges if it is worthwhile doing it?
I have drawn several situations and tried to construct an example in which
using the same payment hash for the JIT-rebalancing would result in a
severe problem with the payment process in the sense that it would be
compromised or somebody could steal funds. It could however be a privacy
issue as more nodes are being aware of the same payment (but that is also
the case with base-AMP)
I was not able to construct such a situation (under the assumption that the
rebalancing amount does not exceed the original payment amount). My feeling
(though I have not done this yet) is that one should be able to proof that
taking the same payment hash would always work and in fact create a
situation in which at least the rebalancing only takes place if the entire
payment was routed successfully.
Assuming someone will be able to proof that using the same payment hash for
JIT Routing is not an issue we still run into another problem (which I
believe can be fixed with another MUST rule but comes with quite some
implementation overhead.)
The deadlock problem when doing JIT Routing with the same payment hash:
When using the same payment hash there will be two htlc's (potentially of
different amounts) in opposing directions on the same channel (or in the
lnd case maybe between separate channels between the same two peers).
Unluckily without a novel rule this can produce a deadlock.
As an example take the situation from my initial email with an additional
recipient R1:
100 / 110 80 / 200 150/180
S ----------> B --------> R ----------> R1
\ /
80/200 \ / 100/200
\ /
T
Meaning we have the following channels:
S ---> B capacity: 110 A funds: 100 B funds: 10
B ---> R capacity: 200 B funds: 80 R funds: 120
B ---> T capacity: 200 B funds: 80 T funds: 120
T ---> R capacity: 200 T funds: 100 R funds: 100
R ---> R1 capacity: 180 R funds: 150 R1 funds: 30
neglecting fees the following htlcs would be offered
1.) S-->B amount: 90
2.) B-->T amount:50
3.) T-->R amount:50
4.) R-->B amount: 50
5.) B-->R amount: 90 (difficult to set up before 4. settles)
6.) R-->R1 amount: 90
while 1,5 and 6 are the original path 2,3,4 are the JIT rebalancing.
We see that in this situation using the same preimage results in a problem.
Since the rebalancing is not settled R will not accept the 5th htlc (B--->R
amount: 90) as there is not enough liquidity on B's side producing a
deadlock
However since the same payment hash is used it is save to combine the 4th
and 5th htlc to have the following situation:
1.) S-->B amount: 90
2.) B-->T amount:50
3.) T-->R amount:50
4.) R-->B will be removed or settled or replaced by the 5th htlcs with a
different amount (90 - 50)
5.) B-->R amount: 40
6.) R-->R1 amount: 90
Note that while theoretically it seems tempting to just have two htlc
outputs as the second node could always claim the htlc if the first claims
theirs. However this will not work onchain as potentially more funds are
spend than exist.
Therefor we need a MUST-rule to fix the deadlock problem (which could
probably also be formulated in a symmetric way):
If a node N offers an htlc to a partner with an amount x from whom the node
already received an htlc y (where y is smaller than x) the nodes must
create a new channel state discarding the old htlc but having a new offer
from N with the amount x-y.
This decreases the liquidity bound in the routing process, enables for a
critical channel to be rebalanced several times during several JIT
operations and keeps the onchain footprint low as there are less htlc
outputs.
Also as mentioned above it seems crucial that the rebalancing amount must
not exceed the original payment amount if the payment hash is being reused.
Imagine there was no R1 and R was actually the final destination and B
decides to rebalance an amount larger than necessary (which could not
happen in our setup). R could release the preimage before setting up the
final htlc from R back to B to fulfill the rebalancing request. This could
also happen if T was the final recipient (which R would not now!)
The only way how I see that this problem can be mitigated is by introducing
the following rule (morally even as a MUST rule)
If a node decides to engage in JIT Routing using the same payment hash as
the incoming htlc it SHOULD NOT rebalance an amount higher than the
incoming HTLCs. If it rebalances with a new payment hash it MAY use an
arbitrary amount.
Rational: (as described above)
Another problem while reusing the payment hash is that in this situation
the node who issued the invoice could be involved in a rebalancing act and
decline an htlc as the amount is not sufficient for the invoice. Therefor
we would have to set either the base-AMP feature flag or create a new one
for JIT-routing which would signal that there are more htlc's coming which
need to be combined to forward an onion.
In order to avoid this complex aggregation of htlcs we could also see this
process as a local AMP right a way saying that a node instead of forwarding
the onion MAY do a local base-AMP to the next recipient.
So in my opinion this suggestions to reuse the payment hash is not only
reasonable but actually desirable in particular if we can proof that it is
not a problem and if we agree that we can mitigate the technical challenges
which I described in this mail.
Best regards Rene
On Thu, Mar 14, 2019 at 2:08 PM Ariel Lorenzo-Luaces <arielluaces at gmail.com>
wrote:
> Hello Rene, ZmnSCPxj, and list
>
>
> I really like the proposal and I'm sure it's the correct way forward for
> reducing payment failures and increasing privacy (through mitigating
> probing based network analysis)
>
>
> However I am concerned that this proposal could introduce a vulnerability
> to a spoofed-payment attack.
>
>
> An adversary could create a malicious payment that's guaranteed to fail,
> for free. Any intermediary nodes on the path of the payment before it fails
> could lose fees due to rebalancing if the rebalancing path's success is not
> dependent on the original payment's success.
>
>
> Could anyone speak to this and confirm whether it would be possible for
> the rebalancing step to reuse the original payment hash? If this is
> possible then it should explicitly be included in this proposal.
>
>
> If reusing the payment hash is not possible on the routing path then JIT
> routing would need some other mitigation for the spoofed-payment attack.
>
>
> Cheers
>
>
> Ariel Lorenzo-Luaces
>
>
> On Mar 14, 2019, at 7:45 AM, ZmnSCPxj via Lightning-dev <
> lightning-dev at lists.linuxfoundation.org> wrote:
>>
>> Good morning Rene and list,
>>
>> Let us consider then the rule *when* a rebalancing would be beneficial to the node.
>>
>> The node is offered a fee amount (`offered_fee_amount`) for the forwarding.
>> It knows that, under current channel states, it will definitely have to fail and earn 0 fees.
>>
>> If it engages in JIT-routing, it must pay some fee (`rebalancing_fee_amount`) for the rebalancing operation.
>> But even if it successfully forwards its hop, it is still possible that the route will fail anyway and it will earn 0 fees.
>>
>> So let us consider the probability of success (`success_rate`), a value between 0 to 1.0.
>> This is the estimated probability that we will succeed the route after we forward it.
>>
>> We should only engage in JIT-routing if:
>>
>> offered_fee_amount * success_rate - rebalancing_fee_amount > 0
>>
>> The LHS of the subtraction is the expected earning, while the RHS of the subtraction is the expected cost.
>> The above is trivial accounting for determining net earnings.
>>
>> The fee amounts are trivially computable.
>> Presumably the rebalancing code can compute the fee given a particular rebalance path, and thus can provide `rebalancing_fee_amount`.
>>
>> The `success_rate` can be computed statically from some node data.
>> Better, is for the node to start with this precomputed static information, but augment this over time with its own experienced `success_rate` for all forwards.
>>
>> Regards,
>> ZmnSCPxj
>> ------------------------------
>>
>> Lightning-dev mailing list
>> Lightning-dev at lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>>
>> _______________________________________________
> Lightning-dev mailing list
> Lightning-dev at lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>
--
https://www.rene-pickhardt.de
Skype: rene.pickhardt
mobile: +49 (0)176 5762 3618
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20190314/35731641/attachment-0001.html>;