Matt Corallo [ARCHIVE] on Nostr: 📅 Original date posted:2021-08-25 📝 Original message: I feel like we're having ...
📅 Original date posted:2021-08-25
📝 Original message:
I feel like we're having two very, very different conversations here. On one hand, you're arguing that the base fee is
of marginal use, and that maybe we can figure out how to average it out such that we can avoid needing it. On the other
hand, I'm arguing that, yes, maybe you can, but ideally you wouldn't have to, because its still pretty nice to capture
those costs sometimes. Also, even if we can maybe do away with the base fee, that still doesn't mean we should start
relying on the lack of any not-completely-linear-in-HTLC-value fees in our routing algorithms, as maybe we'll want to do
upfront payments or some other kind of anti-DoS payment in the future to solve the gaping, glaring, giant DoS hole that
is HTLCs taking forever to time out.
I'm not even sure that you're trying to argue, here, that we should start making key assumptions about the only fee
being a proportional one in our routing algorithms, but that is what the topic at hand is, so I can't help but assume
you are?
If you disagree with the above characterization I'm happy to go line-by-line tit-for-tat, but usually those kinds of
tirades aren't exactly useful and end up being more about semantics than the thrust of the argument.
Matt
On 8/20/21 21:46, Anthony Towns wrote:
> On Mon, Aug 16, 2021 at 12:48:36AM -0400, Matt Corallo wrote:
>>> The base+proportional fees paid only on success roughly match the *value*
>>> of forwarding an HTLC, they don't match the costs particularly well
>>> at all.
>> Sure, indeed, there's some additional costs which are not covered by failed
>> HTLCs, [...]
>> Dropping base fee makes the whole situation a good chunk *worse*.
>
> Can you justify that quantitatively?
>
> Like, pick a realistic scenario, where you can make things profitable
> with some particular base_fee, prop_fee, min_htlc_amount combination,
> but can't reasonably pick another similarly profitable outcome with
> base_fee=0? (You probably need to have a bimodal payment distribution
> with a micropayment peak and a regular payment peak, I guess, or perhaps
> have particularly inelastic demand and highly competitive supply?)
>
>>> And all those costs can be captured equally well (or badly) by just
>>> setting a proportional fee and a minimum payment value. I don't know why
>>> you keep ignoring that point.
>> I didn't ignore this, I just disagree, and I'm not entirely sure why you're ignoring the points I made to that effect :).
>
> I don't think I've seen you explicitly disagree with that previously,
> nor explain why you disagree with it? (If I've missed that, a reference
> appreciated; explicit re-explanation also appreciated)
>
>> In all seriousness, I'm entirely unsure why you think proportional is just
>> as good?
>
> In principle, because fee structures already aren't a good match, and
> a simple approximation is better that a complicated approximation.
> Specifically, because you can set
>
> min_htlc_msat=old_base_fee_msat * 1e6 / prop_fee_millionths
>
> which still ensures every HTLC you forward offers a minimum fee of
> old_base_fee_msat, and your fees still increase as the value transferred
> goes up, which in the current lightning environment seems like it's just
> as good an approximation as if you'd actually used "old_base_fee_msat".
>
> For example, maybe you pay $40/month for your node, which is about 40msat
> per second [0], and you really can only do one HTLC per second on average
> [1]. Then instead of a base_fee of 40msat, pick your proportional rate,
> say 0.03%, and calculate your min_htlc amount as above, ie 133sat. So if
> someone sends 5c/133sat through you, they'll pay 40msat, and for every
> ~3 additional sats, they'll pay you an additional 1msat. Your costs are
> covered, and provided your fee rate is competitive and there's traffic
> on the network, you'll make your desired profit.
>
> If your section of the lightning network is being used mainly for
> microtransactions, and you're not competitive/profitable when limiting
> yourself to >5c transactions, you could increase your proportional fee
> and lower your min_htlc amount, eg to 1% and 4sat so that you'll get
> your 40msat from a 4sat/0.16c HTLC, and increase at a rate of 10msat/sat
> after that.
>
> That at least matches the choices you're probably actually making as a
> node operator: "I'm trying to be cheap at 0.03% and focus on relatively
> large transfers" vs "I'm focussing on microtransactions by reducing the
> minimum amount I'll support and being a bit expensive". I don't think
> anyone's setting a base fee by calculating per-tx costs (and if they
> were, per the footnote, I'm not convinced it'd even justify 1msat let
> alone 1sat per tx).
>
> OTOH, if you want to model an arbitrary concave fee function (because
> you have some scheme that optimises fee income by discriminating against
> smaller payments), you could do that by having multiple channels between
> the same nodes, which is much more effective with (base, prop) fee pairs
> than with (prop, min) pairs. (With (prop, min) pairs, you end up with
> large ranges of the domain that would prefer to pay prop2*min2 rather
> than prop1*x when x<min2) [2]
>
> It's not clear to me that's desirable in practice -- the benefit of a
> concave fee function is that it penalises smaller payments / benefits
> larger payments, which naturally penalises splitting payments up and
> (to some extent) micropayments in general. OTOH, it's whatever the
> "turing complete" version of setting fees is, which does at least feel
> theoretically appealing.
>
>> As you note, the cost for nodes is a function of the opportunity
>> cost of the capital, and opportunity cost of the HTLC slots. Lets say as a
>> routing node I decide that the opportunity cost of one of my HTLC slots is
>> generally 1 sat per second, and the average HTLC is fulfilled in one second.
>> Why is it that a proportional fee captures this "equally well"?!
>
> If I send an HTLC through you, I can pay your 1 sat fee, then keep the
> HTLC open for a day, costing you 86,400 sats by your numbers. So I don't
> think that's even remotely close to capturing the costs of the individual
> HTLC that's paying the fee.
>
> But if your averages are right, and enough people are nice despite me
> being a PITA, then you can get the same minimum with a proportional fee;
> if you're charging 0.1% you set the minimum amount to be 1000 sats.
>
> (But 1sat per HTLC is ridiculously expensive, like at least 20x over
> what your actual costs would be, even if your hardware is horribly slow
> and horribly expensive)
>
>> Yes, you could amortize it,
>
> You're already amortizing it: that's what "generally 1 sat per second"
> and "average HTLC is fulfilled in one second" is capturing.
>
>> but that doesn't make it "equally" good, and
>> there are semi-serious proposals to start ignoring nodes that *dont* set
>> their fees to some particular structure in routing decisions.
>
> Are there proposals to do that outside of experimental plugins? That
> seems... aggressive, particularly given it'd exclude three-quarters of
> the network, and any node running with default parameters.
>
>> Sure, nodes
>> can do what they want, but its kinda absurd to suggest that this is a
>> perfectly fine thing to do absent a somewhat compelling reason. This goes
>> doubly because deploying such things significantly will mean we cannot do
>> future protocol changes which may better capture the time-value of node
>> resources!
>
> I think you're getting a bit of a "consensus" mindset there -- if we
> need to change lightning routing algorithms later, it'll be a pain,
> sure, but old behaviours aren't locked in permanently in the same way
> they are for bitcoin consensus rules.
>
> We'll need an similar network-wide routing upgrade to support PTLCs,
> and also (in my opinion anyway) to support a new fee mechanism that deals
> with failed payments. Also needing one to re-support specifying a base fee
> would be pretty easy by comparison, especially since we've "been there,
> done that".
>
> But I don't think anyone's proposing deploying such things "significantly"
> in the first place?
>
>>>>> Additionally, I don't think HTLC slot usage needs to be kept as a
>>>>> limitation after we switch to eltoo;
>>>> The HTLC slot limit is to keep transactions broadcastable. I don't see why
>>>> this would change, you still get an output for each HTLC on the latest
>>>> commitment in eltoo, AFAIU.
>>> eltoo gives us the ability to have channel factories....
>> That doesn't solve the issue at all - you still have a ton of transactions
>> and transaction outputs and spends thereof to put on the chain in the case
>> of a closure with pending HTLCs.
>
> It solves the broadcastability issue. If you're worrying about ending up
> with 5GB worth of pending HTLCs and not being able to atomically post
> that to the blockchain because their timeouts all expire in less than
> 5000 blocks, then sure, there's still other limits you might want to
> care about.
>
>>> (By "any time soon" I mean, I could see software defaults changing if
>>> over 50% of the network deliberately switched to zero base fees and found
>>> it worked fine; and I could see deprecating non-zero fees if that ended
>>> up with 90% of the network on zero base fees, no good reasons for node
>>> operators wanting to stick with running non-zero base fees, and the
>>> experimental algos that relied on zero base fees being significantly
>>> easier to maintain or faster/better)
>> What is your definition of "works fine" here? In today's
>> nearly-entirely-altruistic-routing-node network, we could probably entirely
>> drop the routing fees and things would "work fine". That doesn't make it a
>> good idea for the long-term health of the network.
>
> "We have a problem, and something must be done. This is something,
> therefore it must be done" ? Fixed fees that apply only to successful
> transactions aren't a solution for the long term health of the
> network. Yes, something to address that must be done, but base fees
> aren't something that addresses it, so IMO it's just not a relevant point.
>
> Many node operators actively rejecting the recommendation would be an
> easy example of not "working fine" -- that's certainly what I'd expect to
> see if the recommendation in question was "just don't charge any fees".
> (Equally, I wouldn't have any problem with a #zerofees twitter campaign
> to encourage people to try it out; though I don't think I'd support that
> one myself. Presumably if there were a reasonable zero fee path between
> sender and recipient most lightning nodes would already try that first?)
>
> Cheers,
> aj
>
> [0] $1/month is about 1/40k BTC/month equals 10e3/4 sat/month;
> there's about 2.5e6 seconds in a month (~28.9 days), so that's
> about 10e3/10e6 sat/second or 1 msat/second. Neat.
>
> [1] A $40/month linode gives you 4 EPYC cores, so really you ought
> to be able to do 100s of payments per second, giving you a per
> payment cost of 0.4msat or less as far as I can see... If you're
> ending up with 40msat after amortizing that's hundreds of failed
> payments per success, if you're ending up at 1sat, that's thousands
> of failed payments per success. (Not sure how the maths changes if
> you're self-hosting or have dedicated hosting or if shared hosting
> doesn't actually provide it's nominal performance)
>
> [2] Dammit, the theoretical argument is kind-of convincing me at this
> point. (Hence the delay in posting this...)
>
📝 Original message:
I feel like we're having two very, very different conversations here. On one hand, you're arguing that the base fee is
of marginal use, and that maybe we can figure out how to average it out such that we can avoid needing it. On the other
hand, I'm arguing that, yes, maybe you can, but ideally you wouldn't have to, because its still pretty nice to capture
those costs sometimes. Also, even if we can maybe do away with the base fee, that still doesn't mean we should start
relying on the lack of any not-completely-linear-in-HTLC-value fees in our routing algorithms, as maybe we'll want to do
upfront payments or some other kind of anti-DoS payment in the future to solve the gaping, glaring, giant DoS hole that
is HTLCs taking forever to time out.
I'm not even sure that you're trying to argue, here, that we should start making key assumptions about the only fee
being a proportional one in our routing algorithms, but that is what the topic at hand is, so I can't help but assume
you are?
If you disagree with the above characterization I'm happy to go line-by-line tit-for-tat, but usually those kinds of
tirades aren't exactly useful and end up being more about semantics than the thrust of the argument.
Matt
On 8/20/21 21:46, Anthony Towns wrote:
> On Mon, Aug 16, 2021 at 12:48:36AM -0400, Matt Corallo wrote:
>>> The base+proportional fees paid only on success roughly match the *value*
>>> of forwarding an HTLC, they don't match the costs particularly well
>>> at all.
>> Sure, indeed, there's some additional costs which are not covered by failed
>> HTLCs, [...]
>> Dropping base fee makes the whole situation a good chunk *worse*.
>
> Can you justify that quantitatively?
>
> Like, pick a realistic scenario, where you can make things profitable
> with some particular base_fee, prop_fee, min_htlc_amount combination,
> but can't reasonably pick another similarly profitable outcome with
> base_fee=0? (You probably need to have a bimodal payment distribution
> with a micropayment peak and a regular payment peak, I guess, or perhaps
> have particularly inelastic demand and highly competitive supply?)
>
>>> And all those costs can be captured equally well (or badly) by just
>>> setting a proportional fee and a minimum payment value. I don't know why
>>> you keep ignoring that point.
>> I didn't ignore this, I just disagree, and I'm not entirely sure why you're ignoring the points I made to that effect :).
>
> I don't think I've seen you explicitly disagree with that previously,
> nor explain why you disagree with it? (If I've missed that, a reference
> appreciated; explicit re-explanation also appreciated)
>
>> In all seriousness, I'm entirely unsure why you think proportional is just
>> as good?
>
> In principle, because fee structures already aren't a good match, and
> a simple approximation is better that a complicated approximation.
> Specifically, because you can set
>
> min_htlc_msat=old_base_fee_msat * 1e6 / prop_fee_millionths
>
> which still ensures every HTLC you forward offers a minimum fee of
> old_base_fee_msat, and your fees still increase as the value transferred
> goes up, which in the current lightning environment seems like it's just
> as good an approximation as if you'd actually used "old_base_fee_msat".
>
> For example, maybe you pay $40/month for your node, which is about 40msat
> per second [0], and you really can only do one HTLC per second on average
> [1]. Then instead of a base_fee of 40msat, pick your proportional rate,
> say 0.03%, and calculate your min_htlc amount as above, ie 133sat. So if
> someone sends 5c/133sat through you, they'll pay 40msat, and for every
> ~3 additional sats, they'll pay you an additional 1msat. Your costs are
> covered, and provided your fee rate is competitive and there's traffic
> on the network, you'll make your desired profit.
>
> If your section of the lightning network is being used mainly for
> microtransactions, and you're not competitive/profitable when limiting
> yourself to >5c transactions, you could increase your proportional fee
> and lower your min_htlc amount, eg to 1% and 4sat so that you'll get
> your 40msat from a 4sat/0.16c HTLC, and increase at a rate of 10msat/sat
> after that.
>
> That at least matches the choices you're probably actually making as a
> node operator: "I'm trying to be cheap at 0.03% and focus on relatively
> large transfers" vs "I'm focussing on microtransactions by reducing the
> minimum amount I'll support and being a bit expensive". I don't think
> anyone's setting a base fee by calculating per-tx costs (and if they
> were, per the footnote, I'm not convinced it'd even justify 1msat let
> alone 1sat per tx).
>
> OTOH, if you want to model an arbitrary concave fee function (because
> you have some scheme that optimises fee income by discriminating against
> smaller payments), you could do that by having multiple channels between
> the same nodes, which is much more effective with (base, prop) fee pairs
> than with (prop, min) pairs. (With (prop, min) pairs, you end up with
> large ranges of the domain that would prefer to pay prop2*min2 rather
> than prop1*x when x<min2) [2]
>
> It's not clear to me that's desirable in practice -- the benefit of a
> concave fee function is that it penalises smaller payments / benefits
> larger payments, which naturally penalises splitting payments up and
> (to some extent) micropayments in general. OTOH, it's whatever the
> "turing complete" version of setting fees is, which does at least feel
> theoretically appealing.
>
>> As you note, the cost for nodes is a function of the opportunity
>> cost of the capital, and opportunity cost of the HTLC slots. Lets say as a
>> routing node I decide that the opportunity cost of one of my HTLC slots is
>> generally 1 sat per second, and the average HTLC is fulfilled in one second.
>> Why is it that a proportional fee captures this "equally well"?!
>
> If I send an HTLC through you, I can pay your 1 sat fee, then keep the
> HTLC open for a day, costing you 86,400 sats by your numbers. So I don't
> think that's even remotely close to capturing the costs of the individual
> HTLC that's paying the fee.
>
> But if your averages are right, and enough people are nice despite me
> being a PITA, then you can get the same minimum with a proportional fee;
> if you're charging 0.1% you set the minimum amount to be 1000 sats.
>
> (But 1sat per HTLC is ridiculously expensive, like at least 20x over
> what your actual costs would be, even if your hardware is horribly slow
> and horribly expensive)
>
>> Yes, you could amortize it,
>
> You're already amortizing it: that's what "generally 1 sat per second"
> and "average HTLC is fulfilled in one second" is capturing.
>
>> but that doesn't make it "equally" good, and
>> there are semi-serious proposals to start ignoring nodes that *dont* set
>> their fees to some particular structure in routing decisions.
>
> Are there proposals to do that outside of experimental plugins? That
> seems... aggressive, particularly given it'd exclude three-quarters of
> the network, and any node running with default parameters.
>
>> Sure, nodes
>> can do what they want, but its kinda absurd to suggest that this is a
>> perfectly fine thing to do absent a somewhat compelling reason. This goes
>> doubly because deploying such things significantly will mean we cannot do
>> future protocol changes which may better capture the time-value of node
>> resources!
>
> I think you're getting a bit of a "consensus" mindset there -- if we
> need to change lightning routing algorithms later, it'll be a pain,
> sure, but old behaviours aren't locked in permanently in the same way
> they are for bitcoin consensus rules.
>
> We'll need an similar network-wide routing upgrade to support PTLCs,
> and also (in my opinion anyway) to support a new fee mechanism that deals
> with failed payments. Also needing one to re-support specifying a base fee
> would be pretty easy by comparison, especially since we've "been there,
> done that".
>
> But I don't think anyone's proposing deploying such things "significantly"
> in the first place?
>
>>>>> Additionally, I don't think HTLC slot usage needs to be kept as a
>>>>> limitation after we switch to eltoo;
>>>> The HTLC slot limit is to keep transactions broadcastable. I don't see why
>>>> this would change, you still get an output for each HTLC on the latest
>>>> commitment in eltoo, AFAIU.
>>> eltoo gives us the ability to have channel factories....
>> That doesn't solve the issue at all - you still have a ton of transactions
>> and transaction outputs and spends thereof to put on the chain in the case
>> of a closure with pending HTLCs.
>
> It solves the broadcastability issue. If you're worrying about ending up
> with 5GB worth of pending HTLCs and not being able to atomically post
> that to the blockchain because their timeouts all expire in less than
> 5000 blocks, then sure, there's still other limits you might want to
> care about.
>
>>> (By "any time soon" I mean, I could see software defaults changing if
>>> over 50% of the network deliberately switched to zero base fees and found
>>> it worked fine; and I could see deprecating non-zero fees if that ended
>>> up with 90% of the network on zero base fees, no good reasons for node
>>> operators wanting to stick with running non-zero base fees, and the
>>> experimental algos that relied on zero base fees being significantly
>>> easier to maintain or faster/better)
>> What is your definition of "works fine" here? In today's
>> nearly-entirely-altruistic-routing-node network, we could probably entirely
>> drop the routing fees and things would "work fine". That doesn't make it a
>> good idea for the long-term health of the network.
>
> "We have a problem, and something must be done. This is something,
> therefore it must be done" ? Fixed fees that apply only to successful
> transactions aren't a solution for the long term health of the
> network. Yes, something to address that must be done, but base fees
> aren't something that addresses it, so IMO it's just not a relevant point.
>
> Many node operators actively rejecting the recommendation would be an
> easy example of not "working fine" -- that's certainly what I'd expect to
> see if the recommendation in question was "just don't charge any fees".
> (Equally, I wouldn't have any problem with a #zerofees twitter campaign
> to encourage people to try it out; though I don't think I'd support that
> one myself. Presumably if there were a reasonable zero fee path between
> sender and recipient most lightning nodes would already try that first?)
>
> Cheers,
> aj
>
> [0] $1/month is about 1/40k BTC/month equals 10e3/4 sat/month;
> there's about 2.5e6 seconds in a month (~28.9 days), so that's
> about 10e3/10e6 sat/second or 1 msat/second. Neat.
>
> [1] A $40/month linode gives you 4 EPYC cores, so really you ought
> to be able to do 100s of payments per second, giving you a per
> payment cost of 0.4msat or less as far as I can see... If you're
> ending up with 40msat after amortizing that's hundreds of failed
> payments per success, if you're ending up at 1sat, that's thousands
> of failed payments per success. (Not sure how the maths changes if
> you're self-hosting or have dedicated hosting or if shared hosting
> doesn't actually provide it's nominal performance)
>
> [2] Dammit, the theoretical argument is kind-of convincing me at this
> point. (Hence the delay in posting this...)
>