Anthony Towns [ARCHIVE] on Nostr: 📅 Original date posted:2021-08-21 📝 Original message: On Mon, Aug 16, 2021 at ...
📅 Original date posted:2021-08-21
📝 Original message:
On Mon, Aug 16, 2021 at 12:48:36AM -0400, Matt Corallo wrote:
> > The base+proportional fees paid only on success roughly match the *value*
> > of forwarding an HTLC, they don't match the costs particularly well
> > at all.
> Sure, indeed, there's some additional costs which are not covered by failed
> HTLCs, [...]
> Dropping base fee makes the whole situation a good chunk *worse*.
Can you justify that quantitatively?
Like, pick a realistic scenario, where you can make things profitable
with some particular base_fee, prop_fee, min_htlc_amount combination,
but can't reasonably pick another similarly profitable outcome with
base_fee=0? (You probably need to have a bimodal payment distribution
with a micropayment peak and a regular payment peak, I guess, or perhaps
have particularly inelastic demand and highly competitive supply?)
> > And all those costs can be captured equally well (or badly) by just
> > setting a proportional fee and a minimum payment value. I don't know why
> > you keep ignoring that point.
> I didn't ignore this, I just disagree, and I'm not entirely sure why you're ignoring the points I made to that effect :).
I don't think I've seen you explicitly disagree with that previously,
nor explain why you disagree with it? (If I've missed that, a reference
appreciated; explicit re-explanation also appreciated)
> In all seriousness, I'm entirely unsure why you think proportional is just
> as good?
In principle, because fee structures already aren't a good match, and
a simple approximation is better that a complicated approximation.
Specifically, because you can set
min_htlc_msat=old_base_fee_msat * 1e6 / prop_fee_millionths
which still ensures every HTLC you forward offers a minimum fee of
old_base_fee_msat, and your fees still increase as the value transferred
goes up, which in the current lightning environment seems like it's just
as good an approximation as if you'd actually used "old_base_fee_msat".
For example, maybe you pay $40/month for your node, which is about 40msat
per second [0], and you really can only do one HTLC per second on average
[1]. Then instead of a base_fee of 40msat, pick your proportional rate,
say 0.03%, and calculate your min_htlc amount as above, ie 133sat. So if
someone sends 5c/133sat through you, they'll pay 40msat, and for every
~3 additional sats, they'll pay you an additional 1msat. Your costs are
covered, and provided your fee rate is competitive and there's traffic
on the network, you'll make your desired profit.
If your section of the lightning network is being used mainly for
microtransactions, and you're not competitive/profitable when limiting
yourself to >5c transactions, you could increase your proportional fee
and lower your min_htlc amount, eg to 1% and 4sat so that you'll get
your 40msat from a 4sat/0.16c HTLC, and increase at a rate of 10msat/sat
after that.
That at least matches the choices you're probably actually making as a
node operator: "I'm trying to be cheap at 0.03% and focus on relatively
large transfers" vs "I'm focussing on microtransactions by reducing the
minimum amount I'll support and being a bit expensive". I don't think
anyone's setting a base fee by calculating per-tx costs (and if they
were, per the footnote, I'm not convinced it'd even justify 1msat let
alone 1sat per tx).
OTOH, if you want to model an arbitrary concave fee function (because
you have some scheme that optimises fee income by discriminating against
smaller payments), you could do that by having multiple channels between
the same nodes, which is much more effective with (base, prop) fee pairs
than with (prop, min) pairs. (With (prop, min) pairs, you end up with
large ranges of the domain that would prefer to pay prop2*min2 rather
than prop1*x when x<min2) [2]
It's not clear to me that's desirable in practice -- the benefit of a
concave fee function is that it penalises smaller payments / benefits
larger payments, which naturally penalises splitting payments up and
(to some extent) micropayments in general. OTOH, it's whatever the
"turing complete" version of setting fees is, which does at least feel
theoretically appealing.
> As you note, the cost for nodes is a function of the opportunity
> cost of the capital, and opportunity cost of the HTLC slots. Lets say as a
> routing node I decide that the opportunity cost of one of my HTLC slots is
> generally 1 sat per second, and the average HTLC is fulfilled in one second.
> Why is it that a proportional fee captures this "equally well"?!
If I send an HTLC through you, I can pay your 1 sat fee, then keep the
HTLC open for a day, costing you 86,400 sats by your numbers. So I don't
think that's even remotely close to capturing the costs of the individual
HTLC that's paying the fee.
But if your averages are right, and enough people are nice despite me
being a PITA, then you can get the same minimum with a proportional fee;
if you're charging 0.1% you set the minimum amount to be 1000 sats.
(But 1sat per HTLC is ridiculously expensive, like at least 20x over
what your actual costs would be, even if your hardware is horribly slow
and horribly expensive)
> Yes, you could amortize it,
You're already amortizing it: that's what "generally 1 sat per second"
and "average HTLC is fulfilled in one second" is capturing.
> but that doesn't make it "equally" good, and
> there are semi-serious proposals to start ignoring nodes that *dont* set
> their fees to some particular structure in routing decisions.
Are there proposals to do that outside of experimental plugins? That
seems... aggressive, particularly given it'd exclude three-quarters of
the network, and any node running with default parameters.
> Sure, nodes
> can do what they want, but its kinda absurd to suggest that this is a
> perfectly fine thing to do absent a somewhat compelling reason. This goes
> doubly because deploying such things significantly will mean we cannot do
> future protocol changes which may better capture the time-value of node
> resources!
I think you're getting a bit of a "consensus" mindset there -- if we
need to change lightning routing algorithms later, it'll be a pain,
sure, but old behaviours aren't locked in permanently in the same way
they are for bitcoin consensus rules.
We'll need an similar network-wide routing upgrade to support PTLCs,
and also (in my opinion anyway) to support a new fee mechanism that deals
with failed payments. Also needing one to re-support specifying a base fee
would be pretty easy by comparison, especially since we've "been there,
done that".
But I don't think anyone's proposing deploying such things "significantly"
in the first place?
> > > > Additionally, I don't think HTLC slot usage needs to be kept as a
> > > > limitation after we switch to eltoo;
> > > The HTLC slot limit is to keep transactions broadcastable. I don't see why
> > > this would change, you still get an output for each HTLC on the latest
> > > commitment in eltoo, AFAIU.
> > eltoo gives us the ability to have channel factories....
> That doesn't solve the issue at all - you still have a ton of transactions
> and transaction outputs and spends thereof to put on the chain in the case
> of a closure with pending HTLCs.
It solves the broadcastability issue. If you're worrying about ending up
with 5GB worth of pending HTLCs and not being able to atomically post
that to the blockchain because their timeouts all expire in less than
5000 blocks, then sure, there's still other limits you might want to
care about.
> > (By "any time soon" I mean, I could see software defaults changing if
> > over 50% of the network deliberately switched to zero base fees and found
> > it worked fine; and I could see deprecating non-zero fees if that ended
> > up with 90% of the network on zero base fees, no good reasons for node
> > operators wanting to stick with running non-zero base fees, and the
> > experimental algos that relied on zero base fees being significantly
> > easier to maintain or faster/better)
> What is your definition of "works fine" here? In today's
> nearly-entirely-altruistic-routing-node network, we could probably entirely
> drop the routing fees and things would "work fine". That doesn't make it a
> good idea for the long-term health of the network.
"We have a problem, and something must be done. This is something,
therefore it must be done" ? Fixed fees that apply only to successful
transactions aren't a solution for the long term health of the
network. Yes, something to address that must be done, but base fees
aren't something that addresses it, so IMO it's just not a relevant point.
Many node operators actively rejecting the recommendation would be an
easy example of not "working fine" -- that's certainly what I'd expect to
see if the recommendation in question was "just don't charge any fees".
(Equally, I wouldn't have any problem with a #zerofees twitter campaign
to encourage people to try it out; though I don't think I'd support that
one myself. Presumably if there were a reasonable zero fee path between
sender and recipient most lightning nodes would already try that first?)
Cheers,
aj
[0] $1/month is about 1/40k BTC/month equals 10e3/4 sat/month;
there's about 2.5e6 seconds in a month (~28.9 days), so that's
about 10e3/10e6 sat/second or 1 msat/second. Neat.
[1] A $40/month linode gives you 4 EPYC cores, so really you ought
to be able to do 100s of payments per second, giving you a per
payment cost of 0.4msat or less as far as I can see... If you're
ending up with 40msat after amortizing that's hundreds of failed
payments per success, if you're ending up at 1sat, that's thousands
of failed payments per success. (Not sure how the maths changes if
you're self-hosting or have dedicated hosting or if shared hosting
doesn't actually provide it's nominal performance)
[2] Dammit, the theoretical argument is kind-of convincing me at this
point. (Hence the delay in posting this...)
📝 Original message:
On Mon, Aug 16, 2021 at 12:48:36AM -0400, Matt Corallo wrote:
> > The base+proportional fees paid only on success roughly match the *value*
> > of forwarding an HTLC, they don't match the costs particularly well
> > at all.
> Sure, indeed, there's some additional costs which are not covered by failed
> HTLCs, [...]
> Dropping base fee makes the whole situation a good chunk *worse*.
Can you justify that quantitatively?
Like, pick a realistic scenario, where you can make things profitable
with some particular base_fee, prop_fee, min_htlc_amount combination,
but can't reasonably pick another similarly profitable outcome with
base_fee=0? (You probably need to have a bimodal payment distribution
with a micropayment peak and a regular payment peak, I guess, or perhaps
have particularly inelastic demand and highly competitive supply?)
> > And all those costs can be captured equally well (or badly) by just
> > setting a proportional fee and a minimum payment value. I don't know why
> > you keep ignoring that point.
> I didn't ignore this, I just disagree, and I'm not entirely sure why you're ignoring the points I made to that effect :).
I don't think I've seen you explicitly disagree with that previously,
nor explain why you disagree with it? (If I've missed that, a reference
appreciated; explicit re-explanation also appreciated)
> In all seriousness, I'm entirely unsure why you think proportional is just
> as good?
In principle, because fee structures already aren't a good match, and
a simple approximation is better that a complicated approximation.
Specifically, because you can set
min_htlc_msat=old_base_fee_msat * 1e6 / prop_fee_millionths
which still ensures every HTLC you forward offers a minimum fee of
old_base_fee_msat, and your fees still increase as the value transferred
goes up, which in the current lightning environment seems like it's just
as good an approximation as if you'd actually used "old_base_fee_msat".
For example, maybe you pay $40/month for your node, which is about 40msat
per second [0], and you really can only do one HTLC per second on average
[1]. Then instead of a base_fee of 40msat, pick your proportional rate,
say 0.03%, and calculate your min_htlc amount as above, ie 133sat. So if
someone sends 5c/133sat through you, they'll pay 40msat, and for every
~3 additional sats, they'll pay you an additional 1msat. Your costs are
covered, and provided your fee rate is competitive and there's traffic
on the network, you'll make your desired profit.
If your section of the lightning network is being used mainly for
microtransactions, and you're not competitive/profitable when limiting
yourself to >5c transactions, you could increase your proportional fee
and lower your min_htlc amount, eg to 1% and 4sat so that you'll get
your 40msat from a 4sat/0.16c HTLC, and increase at a rate of 10msat/sat
after that.
That at least matches the choices you're probably actually making as a
node operator: "I'm trying to be cheap at 0.03% and focus on relatively
large transfers" vs "I'm focussing on microtransactions by reducing the
minimum amount I'll support and being a bit expensive". I don't think
anyone's setting a base fee by calculating per-tx costs (and if they
were, per the footnote, I'm not convinced it'd even justify 1msat let
alone 1sat per tx).
OTOH, if you want to model an arbitrary concave fee function (because
you have some scheme that optimises fee income by discriminating against
smaller payments), you could do that by having multiple channels between
the same nodes, which is much more effective with (base, prop) fee pairs
than with (prop, min) pairs. (With (prop, min) pairs, you end up with
large ranges of the domain that would prefer to pay prop2*min2 rather
than prop1*x when x<min2) [2]
It's not clear to me that's desirable in practice -- the benefit of a
concave fee function is that it penalises smaller payments / benefits
larger payments, which naturally penalises splitting payments up and
(to some extent) micropayments in general. OTOH, it's whatever the
"turing complete" version of setting fees is, which does at least feel
theoretically appealing.
> As you note, the cost for nodes is a function of the opportunity
> cost of the capital, and opportunity cost of the HTLC slots. Lets say as a
> routing node I decide that the opportunity cost of one of my HTLC slots is
> generally 1 sat per second, and the average HTLC is fulfilled in one second.
> Why is it that a proportional fee captures this "equally well"?!
If I send an HTLC through you, I can pay your 1 sat fee, then keep the
HTLC open for a day, costing you 86,400 sats by your numbers. So I don't
think that's even remotely close to capturing the costs of the individual
HTLC that's paying the fee.
But if your averages are right, and enough people are nice despite me
being a PITA, then you can get the same minimum with a proportional fee;
if you're charging 0.1% you set the minimum amount to be 1000 sats.
(But 1sat per HTLC is ridiculously expensive, like at least 20x over
what your actual costs would be, even if your hardware is horribly slow
and horribly expensive)
> Yes, you could amortize it,
You're already amortizing it: that's what "generally 1 sat per second"
and "average HTLC is fulfilled in one second" is capturing.
> but that doesn't make it "equally" good, and
> there are semi-serious proposals to start ignoring nodes that *dont* set
> their fees to some particular structure in routing decisions.
Are there proposals to do that outside of experimental plugins? That
seems... aggressive, particularly given it'd exclude three-quarters of
the network, and any node running with default parameters.
> Sure, nodes
> can do what they want, but its kinda absurd to suggest that this is a
> perfectly fine thing to do absent a somewhat compelling reason. This goes
> doubly because deploying such things significantly will mean we cannot do
> future protocol changes which may better capture the time-value of node
> resources!
I think you're getting a bit of a "consensus" mindset there -- if we
need to change lightning routing algorithms later, it'll be a pain,
sure, but old behaviours aren't locked in permanently in the same way
they are for bitcoin consensus rules.
We'll need an similar network-wide routing upgrade to support PTLCs,
and also (in my opinion anyway) to support a new fee mechanism that deals
with failed payments. Also needing one to re-support specifying a base fee
would be pretty easy by comparison, especially since we've "been there,
done that".
But I don't think anyone's proposing deploying such things "significantly"
in the first place?
> > > > Additionally, I don't think HTLC slot usage needs to be kept as a
> > > > limitation after we switch to eltoo;
> > > The HTLC slot limit is to keep transactions broadcastable. I don't see why
> > > this would change, you still get an output for each HTLC on the latest
> > > commitment in eltoo, AFAIU.
> > eltoo gives us the ability to have channel factories....
> That doesn't solve the issue at all - you still have a ton of transactions
> and transaction outputs and spends thereof to put on the chain in the case
> of a closure with pending HTLCs.
It solves the broadcastability issue. If you're worrying about ending up
with 5GB worth of pending HTLCs and not being able to atomically post
that to the blockchain because their timeouts all expire in less than
5000 blocks, then sure, there's still other limits you might want to
care about.
> > (By "any time soon" I mean, I could see software defaults changing if
> > over 50% of the network deliberately switched to zero base fees and found
> > it worked fine; and I could see deprecating non-zero fees if that ended
> > up with 90% of the network on zero base fees, no good reasons for node
> > operators wanting to stick with running non-zero base fees, and the
> > experimental algos that relied on zero base fees being significantly
> > easier to maintain or faster/better)
> What is your definition of "works fine" here? In today's
> nearly-entirely-altruistic-routing-node network, we could probably entirely
> drop the routing fees and things would "work fine". That doesn't make it a
> good idea for the long-term health of the network.
"We have a problem, and something must be done. This is something,
therefore it must be done" ? Fixed fees that apply only to successful
transactions aren't a solution for the long term health of the
network. Yes, something to address that must be done, but base fees
aren't something that addresses it, so IMO it's just not a relevant point.
Many node operators actively rejecting the recommendation would be an
easy example of not "working fine" -- that's certainly what I'd expect to
see if the recommendation in question was "just don't charge any fees".
(Equally, I wouldn't have any problem with a #zerofees twitter campaign
to encourage people to try it out; though I don't think I'd support that
one myself. Presumably if there were a reasonable zero fee path between
sender and recipient most lightning nodes would already try that first?)
Cheers,
aj
[0] $1/month is about 1/40k BTC/month equals 10e3/4 sat/month;
there's about 2.5e6 seconds in a month (~28.9 days), so that's
about 10e3/10e6 sat/second or 1 msat/second. Neat.
[1] A $40/month linode gives you 4 EPYC cores, so really you ought
to be able to do 100s of payments per second, giving you a per
payment cost of 0.4msat or less as far as I can see... If you're
ending up with 40msat after amortizing that's hundreds of failed
payments per success, if you're ending up at 1sat, that's thousands
of failed payments per success. (Not sure how the maths changes if
you're self-hosting or have dedicated hosting or if shared hosting
doesn't actually provide it's nominal performance)
[2] Dammit, the theoretical argument is kind-of convincing me at this
point. (Hence the delay in posting this...)