Rusty Russell [ARCHIVE] on Nostr: 📅 Original date posted:2022-04-28 📝 Original message: Matt Corallo <lf-lists at ...
📅 Original date posted:2022-04-28
📝 Original message:
Matt Corallo <lf-lists at mattcorallo.com> writes:
> On 4/26/22 11:53 PM, Rusty Russell wrote:
>> Matt Corallo <lf-lists at mattcorallo.com> writes:
>>>> This same problem will occur if *anyone* does ratelimiting, unless
>>>> *everyone* does. And with minisketch, there's a good reason to do so.
>>>
>>> None of this seems like a good argument for *not* taking the "send updates since the last sync in
>>> the minisketch" approach to reduce the damage inconsistent policies
>>> cause, though?
>>
>> You can't do this, with minisketch. You end up having to keep all the
>> ratelimited differences you're ignoring *per peer*, and then cancelling
>> them out of the minisketch on every receive or send.
>
> Hmm? I'm a bit confused, let me attempt to restate to make sure we're on the same page. What I
> *think* you said here is: "If you have a node which is rejecting a large percentage *channel*'s
> updates (on a per-channel, not per-update basis), and it tries to sync, you'll end up having to keep
> some huge set of 'I dont want any more updates for that channel' on a per-peer basis"? Or maybe you
> might have said "When you rate-limit, you have to tell your peer that you rate-limited a channel
> update and that it shouldn't add that update to its next sketch"?
OK, let's step back. Unlike Bitcoin, we can use a single sketch for
*all* peers. This is because we *can* encode enough information that
you can get useful info from the 64 bit id, and because it's expensive
to create them so you can't spam.
The more boutique per-peer handling we need, the further it gets from
this ideal;.
> The second potential thing I think you might have meant here I don't see as an issue at all? You can
> simply...let the sketch include one channel update that you ignored? See above discussion of similar
> rate-limits.
No, you need to get all the ignored ones somehow? There's so much cruft
in the sketch you can't decode it. Now you need to remember the ones
you ratelimited, and try to match other's ratelimiting.
>> So you end up doing that LND and core-lightning do, which is "pick 3
>> peers to gossip with" and tell everyone else to shut up.
>>
>> Yet the point of minisketch is robustness; you can (at cost of 1 message
>> per minute) keep in sync with an arbitrary number of peers.
>>
>> So, we might as well define a preferred ratelimit, so nodes know that
>> spamming past a certain point is not going to propagate. At the moment,
>> LND has no effective ratelimit at all, so it's a race to the bottom.
>
> I agree there should be *some* rough consensus, but rate-limits are a locally-enforced thing, not a
> global one. There will always be races and updates you reject that your peers dont, no matter the
> rate-limit, and while I agree we should have guidelines, we can't "just make them the same" - it
> both doesn't solve the problem and means we can't change them in the future.
Sure it does! It severly limits the set divergence to race conditions
(down to block height divergence, in practice).
> Ultimately, a updates-based sync is more robust in such a case - if there's some race and your peer
> accepts something you don't it may mean one more entry in the sketch one time, but it won't hang
> around forever.
>
>> We need that limit eventually, this just makes it more of a priority.
>>
>>> I'm not really
>>> sure in a world where you do "update-based-sketch" gossip sync you're any worse off than today even
>>> with different rate-limit policies, though I obviously agree there are substantial issues with the
>>> massively inconsistent rate-limit policies we see today.
>>
>> You can't really do it, since rate-limited junk overwhelms the sketch
>> really fast :(
>
> How is this any better in a non-update-based-sketch? The only way to address it is to have a bigger
> sketch, which you can do no matter the thing you're building the sketch over.
>
> Maybe lets schedule a call to get on the same page, throwing text at each other will likely not move
> very quickly.
Maybe. What's a "non-update" based sketch? Some huge percentage of
gossip is channel_update, so it's kind of the thing we want?
Cheers,
Rusty.
📝 Original message:
Matt Corallo <lf-lists at mattcorallo.com> writes:
> On 4/26/22 11:53 PM, Rusty Russell wrote:
>> Matt Corallo <lf-lists at mattcorallo.com> writes:
>>>> This same problem will occur if *anyone* does ratelimiting, unless
>>>> *everyone* does. And with minisketch, there's a good reason to do so.
>>>
>>> None of this seems like a good argument for *not* taking the "send updates since the last sync in
>>> the minisketch" approach to reduce the damage inconsistent policies
>>> cause, though?
>>
>> You can't do this, with minisketch. You end up having to keep all the
>> ratelimited differences you're ignoring *per peer*, and then cancelling
>> them out of the minisketch on every receive or send.
>
> Hmm? I'm a bit confused, let me attempt to restate to make sure we're on the same page. What I
> *think* you said here is: "If you have a node which is rejecting a large percentage *channel*'s
> updates (on a per-channel, not per-update basis), and it tries to sync, you'll end up having to keep
> some huge set of 'I dont want any more updates for that channel' on a per-peer basis"? Or maybe you
> might have said "When you rate-limit, you have to tell your peer that you rate-limited a channel
> update and that it shouldn't add that update to its next sketch"?
OK, let's step back. Unlike Bitcoin, we can use a single sketch for
*all* peers. This is because we *can* encode enough information that
you can get useful info from the 64 bit id, and because it's expensive
to create them so you can't spam.
The more boutique per-peer handling we need, the further it gets from
this ideal;.
> The second potential thing I think you might have meant here I don't see as an issue at all? You can
> simply...let the sketch include one channel update that you ignored? See above discussion of similar
> rate-limits.
No, you need to get all the ignored ones somehow? There's so much cruft
in the sketch you can't decode it. Now you need to remember the ones
you ratelimited, and try to match other's ratelimiting.
>> So you end up doing that LND and core-lightning do, which is "pick 3
>> peers to gossip with" and tell everyone else to shut up.
>>
>> Yet the point of minisketch is robustness; you can (at cost of 1 message
>> per minute) keep in sync with an arbitrary number of peers.
>>
>> So, we might as well define a preferred ratelimit, so nodes know that
>> spamming past a certain point is not going to propagate. At the moment,
>> LND has no effective ratelimit at all, so it's a race to the bottom.
>
> I agree there should be *some* rough consensus, but rate-limits are a locally-enforced thing, not a
> global one. There will always be races and updates you reject that your peers dont, no matter the
> rate-limit, and while I agree we should have guidelines, we can't "just make them the same" - it
> both doesn't solve the problem and means we can't change them in the future.
Sure it does! It severly limits the set divergence to race conditions
(down to block height divergence, in practice).
> Ultimately, a updates-based sync is more robust in such a case - if there's some race and your peer
> accepts something you don't it may mean one more entry in the sketch one time, but it won't hang
> around forever.
>
>> We need that limit eventually, this just makes it more of a priority.
>>
>>> I'm not really
>>> sure in a world where you do "update-based-sketch" gossip sync you're any worse off than today even
>>> with different rate-limit policies, though I obviously agree there are substantial issues with the
>>> massively inconsistent rate-limit policies we see today.
>>
>> You can't really do it, since rate-limited junk overwhelms the sketch
>> really fast :(
>
> How is this any better in a non-update-based-sketch? The only way to address it is to have a bigger
> sketch, which you can do no matter the thing you're building the sketch over.
>
> Maybe lets schedule a call to get on the same page, throwing text at each other will likely not move
> very quickly.
Maybe. What's a "non-update" based sketch? Some huge percentage of
gossip is channel_update, so it's kind of the thing we want?
Cheers,
Rusty.