Anthony Towns [ARCHIVE] on Nostr: 📅 Original date posted:2022-09-29 📝 Original message: On Thu, Sep 29, 2022 at ...
📅 Original date posted:2022-09-29
📝 Original message:
On Thu, Sep 29, 2022 at 12:41:44AM +0000, ZmnSCPxj wrote:
> > I get what you're saying, but I don't think a "stock of liquidity"
> > is a helpful metaphor/mental model here.
> > "Liquidity" usually means "how easy it is to exchange X for Y" -- assets
> > for cash, etc; but for lightning, liquidity is guaranteed by being
> > able to drop to chain. Likewise, "running out of stock" isn't usually
> > something that gets automatically fixed by someone else coming in and
> > buying something different.
> Semantics.
> You got what I am saying anyway.
Semantics are important. If you choose the wrong analogies, you'll jump
to the wrong conclusions, which I think you're doing here.
> So let me invent a completely new term derived from my local `/dev/random`, "IPpvHg".
If you're going to make up words, at least make them pronouncable...
apt-get install pwgen; pwgen -0A maybe. But there's no need to make
up words; these aren't completely novel concepts, and existing terms
describe the concepts pretty well.
> A patient and rich forwarding node can buy out the IPpvHg stock of many cheaper nodes,
I just spent a lot of words explaining why I disagree with that claim.
Restating it doesn't really seem constructive.
> and that I think is what we are mostly seeing in the network.
I don't really agree. I think we're seeing a combination of unbalanced
overall flows due to an insufficiently circular economy (which would
perhaps be eased by more custodial wallets/exchanges supporting lightning)
and the combination of a lack of any way to limit channel flow other
than raising fees and an inability to dynamically change fees on a
minute-by-minute timescale.
> > (Also, you don't earn 0 profit on an imbalanced channel; you're just
> > forced to stop accepting some txs. Every time you forward $1 in the
> > available direction, you become able to forward another $1 back in the
> > saturated direction; and you earn fees on both those $1s)
> But that is based on the existence of a stock of IPpvHg in another channel.
No, it's not. It applies even if there is only one channel in the
entire network (though I guess that channel would have to be between two
custodial entities, or there wouldn't be any point charging fees in the
first place).
> Actual forwarding node operators classify their peers as "mostly a source" and "mostly a drain" and "mostly balanced", they want CLBOSS to classify peers similarly.
"mostly a source" should trigger rate limiting in one direction, "mostly
a drain" should trigger rate limiting in the other. Both should only be
true briefly, until the rate limiting kicks in and the channel becomes
"mostly balanced".
That's still the case even if the rate limiting is "oops, one side of
the channel has ~0 balance".
> > I think it's better to think in terms of "payment flow" -- are you
> > forwarding $5/hour in one direction, but $10/hour in the other? Is
> > that an ongoing imbalance, or something that evens itself out over time
> > ($120/day in both directions)?
> It is helpful to notice that the channel balance is the integral of the sum of payment flows in both directions.
The channel balance is the sum of the initial balance and all payments,
sure. No need to add an integral in there as well. For a successful,
long lasting channel, sum(incoming payments) and sum(outgoing payments)
will be much greater than the balance, to the point where the balance
is just a rounding error by comparison.
> This is why actual forwarding node operators are obsessed with channel balance.
> They already *are* thinking in terms of payment flow, and using an analytical technique to keep track of it: the channel balance itself.
This is exactly backwards: you don't monitor your profits by looking at
the rounding errors, you monitor your profits by looking at your sales.
If you've forwarded $100,000 in one direction, and $100,200 in the other
direction, you care about the $200,200 total that you were charging fees
on, not the $200 net delta that it's made to your channel balance.
> > Once you start in that direction, there's also a few other questions
> > you can ask:
> >
> > * can I get make more revenue by getting more payment flow at a
> > lower fee, or by charging a higher fee over less payment flow?
>
> As I pointed out, if you sell your stock of IPpvHg at too low a price point, other forwarding nodes will snatch up the cheap IPpvHg, buying out that stock.
> They can then form an effective cartel, selling the stock of IPpvHg at a higher price later.
No; changing your fee rate isn't about messing with other people's
channels, it's about encouraging more use of lightning overall. For
example, if you're charging a base_fee of 1sat per HTLC, then dropping
that to 0sat might reduce fees from existing traffic, but maybe it will
allow you to forward so many micropayments or AMP payments that it's
wortwhile anyway.
The lightning network is tiny; if you're constantly thinking about how
to steal what little profits there are from other participants, instead
of how to grow the ecosystem, you're going to have a really bad time.
(Is AMP Atomic MultiPath, or Atomic Multipath Payment? I prefer the
former, and that's why I don't think I'm saying ATM machine)
> This all implies to me that the main problem might not be *local* imbalances, but *global* ones.
I mean, sure, that seems totally plausible. This thread is about solving
routing problems that will occur even when there *isn't* a global
imbalance of payment flow, though...
> * A node whose channels have depleted contacts one of the payees it often pays to.
> * The first node constructs an onchain HTLC offer to the second node.
> * The second node routes back a payment over the Lightning Network to the first node.
> * The first node releases the payment preimage over Lightning.
> * The second node claims the onchain HTLC via the payment preimage.
> I proposed that as an alternative to splicing.
Isn't this the same as Lightning Labs' Loop idea? It's a fine idea,
but it's not really an alternative to splicing.
> Channel balance is private information.
I mean, it would be nice if it was, but it's not, and the fee rate card
proposal creates a strong incentive for everyone to try to discover
that information?
> > * maybe you and they have different demand at different times of the
> > day, so time-shifting imbalances is mutually beneficial? I don't
> > see how this works out without secret information though -- the
> > people who want to route payments should be choosing the cheapest
> > link at all times of the day already
> Channel balance *is* the private information you are looking for here: it is the integral of the sum of payment flows.
You have precise knowledge of the payment flow for your own channels
already: you run the channel and saw all the payments going each way,
including the ones that failed. You don't need a proxy.
When I say "secret information" I mean that you, as someone who's going
to rebalance, needs to have information about the policies of the other
channel (the one you're going to rebalance over) that the rest of the
network isn't privy too.
> Sure, a remote node through which you rebalance might not share this information to you,
If the rebalance is mutually beneficial (as assumed), then they'll be
happy to share the information with you. But in that case, they'll
probably make the information fully public, at which point people will
just route payments through them instead of you anyway.
> Yes, but again: the integral of payment flows is the channel balance, and you can replace every reference to "flow" here with "channel balance".
Sure. Work out your profits based on the $200 delta, rather than the
$200k of flows...
> > But the same scenario applies in the max_msat world too, with only
> > slight modifications. [...]
> But the same thing can be done in a world where fees are the valve,
I mean, I was duplicating a scenario that you were deriving from a world
where people only use fees, so, obviously?
If you use the other scenario -- with $5 payments, two cheap channels
that insist on balanced flows, one expensive one that doesn't, and an
overall unbalanced flow -- then that doesn't work well if you only have
fees. In that scenario:
* people see two channels with 0.001% fees, pick one at random and
route their entire $5 transaction through it
* if they're lucky, and a $1000 payment has recently gone the other
way, they succed and pay $0.00005 in tx fees
* if they're not (10% of the time), then both the cheap channels will
be drained, and they'll route their $5 payment through the expensive
channel, suffering both latency while they find a workable path, and
paying much higher fees ($0.02)
ie, some people are paying much higher fees (400x higher than the 90%
of lucky people, or ~10x higher than everyone in the max_msat scenario).
> > I don't think it's ideal if a world that includes both:
> > * altruists, who'll forward your payments for cheap/free
> > * profiteers, who are trying to make a living offering lightning
> > services
> In the world outside of Lightning, we usually make the stink eye at any profiteers who exploit altruists and make noises about how such profiteers are evil, and occasionally even punish them a little bit.
Mmm. You (and CLBOSS) are the profiteer exploiting altruists with all
your aggressive rebalancing ideas in this scenario. Be careful what you
wish for maybe?
In a normally functioning economy, you just find the altruists come up
with some way other than prices to limit who they "sell" too -- whether
that be first come first served, or charities only giving alms to the
obviously needy, or clubs that only provide benefits to members, etc.
You could think of max_msat throttling in exactly that role. But, if it
can be made to work, it's more broadly useful than that, since what it's
fundamentally about is avoiding the "whoops, this channel's depleted,
better retry routing via some other path" cases which applies even in
a world without CLBOSS rebalancing (ie, rationing via the "first come
first served" principle).
> > I mean, that's already a fundamental problem with max_msat: why wouldn't
> > payment initiators do exactly that in the first place?
> Indeed, which suggests that the use of `htlc_max_msat` for a flow valve is flawed:
It was first proposed barely a week ago, of course it's flawed. The
question is what the flaws are, and whether they're fundamental or
things that can be worked around.
> it requires continuous payment flow / payment size curve, and the payment size can be manipulated trivially by any third parties.
The blog post was based on a markov model, which has discrete payment
steps by its nature, so I don't know what you're talking about with a
"continuous" payment flow curve. Both "all payments are at the max size
(simplified as 1 unit)" and "payments are uniform between (0, max)" were
modelled, the former seeming a reasonable match to the "manipulation"
you're proposing, at least for a first brush on the topic.
> > I wonder whether a (per source/dest channel?) token bucket rate limit
> > might suffice, though. [...]
> Given that a published channel is a global resource, any rate limit is going to be shared amongst many users, and if you underquote the value of the IPpvHg you are providing, rebalancers are going to grab as much of the rate limit as they can.
The goal here isn't to stop rebalancing, it's to achieve balanced payment
flow by reducing payment attempts, so that fewer payment attempts
fail. (Rather than achieving balanced payment flows by having many
payment attempts fail)
> I think this implies we need to have a mechanism to move funds outside of the Lightning network, i.e. every published node should have onchain/offchain swap capability.
I guess it would be appealing to a profiteer to have a way of forcing
nodes operating competing channels to taint their wallets by sending to
OFAC banned addresses... Sure seems like the sort of feature that should
be opt-in only to me though.
Cheers,
aj
📝 Original message:
On Thu, Sep 29, 2022 at 12:41:44AM +0000, ZmnSCPxj wrote:
> > I get what you're saying, but I don't think a "stock of liquidity"
> > is a helpful metaphor/mental model here.
> > "Liquidity" usually means "how easy it is to exchange X for Y" -- assets
> > for cash, etc; but for lightning, liquidity is guaranteed by being
> > able to drop to chain. Likewise, "running out of stock" isn't usually
> > something that gets automatically fixed by someone else coming in and
> > buying something different.
> Semantics.
> You got what I am saying anyway.
Semantics are important. If you choose the wrong analogies, you'll jump
to the wrong conclusions, which I think you're doing here.
> So let me invent a completely new term derived from my local `/dev/random`, "IPpvHg".
If you're going to make up words, at least make them pronouncable...
apt-get install pwgen; pwgen -0A maybe. But there's no need to make
up words; these aren't completely novel concepts, and existing terms
describe the concepts pretty well.
> A patient and rich forwarding node can buy out the IPpvHg stock of many cheaper nodes,
I just spent a lot of words explaining why I disagree with that claim.
Restating it doesn't really seem constructive.
> and that I think is what we are mostly seeing in the network.
I don't really agree. I think we're seeing a combination of unbalanced
overall flows due to an insufficiently circular economy (which would
perhaps be eased by more custodial wallets/exchanges supporting lightning)
and the combination of a lack of any way to limit channel flow other
than raising fees and an inability to dynamically change fees on a
minute-by-minute timescale.
> > (Also, you don't earn 0 profit on an imbalanced channel; you're just
> > forced to stop accepting some txs. Every time you forward $1 in the
> > available direction, you become able to forward another $1 back in the
> > saturated direction; and you earn fees on both those $1s)
> But that is based on the existence of a stock of IPpvHg in another channel.
No, it's not. It applies even if there is only one channel in the
entire network (though I guess that channel would have to be between two
custodial entities, or there wouldn't be any point charging fees in the
first place).
> Actual forwarding node operators classify their peers as "mostly a source" and "mostly a drain" and "mostly balanced", they want CLBOSS to classify peers similarly.
"mostly a source" should trigger rate limiting in one direction, "mostly
a drain" should trigger rate limiting in the other. Both should only be
true briefly, until the rate limiting kicks in and the channel becomes
"mostly balanced".
That's still the case even if the rate limiting is "oops, one side of
the channel has ~0 balance".
> > I think it's better to think in terms of "payment flow" -- are you
> > forwarding $5/hour in one direction, but $10/hour in the other? Is
> > that an ongoing imbalance, or something that evens itself out over time
> > ($120/day in both directions)?
> It is helpful to notice that the channel balance is the integral of the sum of payment flows in both directions.
The channel balance is the sum of the initial balance and all payments,
sure. No need to add an integral in there as well. For a successful,
long lasting channel, sum(incoming payments) and sum(outgoing payments)
will be much greater than the balance, to the point where the balance
is just a rounding error by comparison.
> This is why actual forwarding node operators are obsessed with channel balance.
> They already *are* thinking in terms of payment flow, and using an analytical technique to keep track of it: the channel balance itself.
This is exactly backwards: you don't monitor your profits by looking at
the rounding errors, you monitor your profits by looking at your sales.
If you've forwarded $100,000 in one direction, and $100,200 in the other
direction, you care about the $200,200 total that you were charging fees
on, not the $200 net delta that it's made to your channel balance.
> > Once you start in that direction, there's also a few other questions
> > you can ask:
> >
> > * can I get make more revenue by getting more payment flow at a
> > lower fee, or by charging a higher fee over less payment flow?
>
> As I pointed out, if you sell your stock of IPpvHg at too low a price point, other forwarding nodes will snatch up the cheap IPpvHg, buying out that stock.
> They can then form an effective cartel, selling the stock of IPpvHg at a higher price later.
No; changing your fee rate isn't about messing with other people's
channels, it's about encouraging more use of lightning overall. For
example, if you're charging a base_fee of 1sat per HTLC, then dropping
that to 0sat might reduce fees from existing traffic, but maybe it will
allow you to forward so many micropayments or AMP payments that it's
wortwhile anyway.
The lightning network is tiny; if you're constantly thinking about how
to steal what little profits there are from other participants, instead
of how to grow the ecosystem, you're going to have a really bad time.
(Is AMP Atomic MultiPath, or Atomic Multipath Payment? I prefer the
former, and that's why I don't think I'm saying ATM machine)
> This all implies to me that the main problem might not be *local* imbalances, but *global* ones.
I mean, sure, that seems totally plausible. This thread is about solving
routing problems that will occur even when there *isn't* a global
imbalance of payment flow, though...
> * A node whose channels have depleted contacts one of the payees it often pays to.
> * The first node constructs an onchain HTLC offer to the second node.
> * The second node routes back a payment over the Lightning Network to the first node.
> * The first node releases the payment preimage over Lightning.
> * The second node claims the onchain HTLC via the payment preimage.
> I proposed that as an alternative to splicing.
Isn't this the same as Lightning Labs' Loop idea? It's a fine idea,
but it's not really an alternative to splicing.
> Channel balance is private information.
I mean, it would be nice if it was, but it's not, and the fee rate card
proposal creates a strong incentive for everyone to try to discover
that information?
> > * maybe you and they have different demand at different times of the
> > day, so time-shifting imbalances is mutually beneficial? I don't
> > see how this works out without secret information though -- the
> > people who want to route payments should be choosing the cheapest
> > link at all times of the day already
> Channel balance *is* the private information you are looking for here: it is the integral of the sum of payment flows.
You have precise knowledge of the payment flow for your own channels
already: you run the channel and saw all the payments going each way,
including the ones that failed. You don't need a proxy.
When I say "secret information" I mean that you, as someone who's going
to rebalance, needs to have information about the policies of the other
channel (the one you're going to rebalance over) that the rest of the
network isn't privy too.
> Sure, a remote node through which you rebalance might not share this information to you,
If the rebalance is mutually beneficial (as assumed), then they'll be
happy to share the information with you. But in that case, they'll
probably make the information fully public, at which point people will
just route payments through them instead of you anyway.
> Yes, but again: the integral of payment flows is the channel balance, and you can replace every reference to "flow" here with "channel balance".
Sure. Work out your profits based on the $200 delta, rather than the
$200k of flows...
> > But the same scenario applies in the max_msat world too, with only
> > slight modifications. [...]
> But the same thing can be done in a world where fees are the valve,
I mean, I was duplicating a scenario that you were deriving from a world
where people only use fees, so, obviously?
If you use the other scenario -- with $5 payments, two cheap channels
that insist on balanced flows, one expensive one that doesn't, and an
overall unbalanced flow -- then that doesn't work well if you only have
fees. In that scenario:
* people see two channels with 0.001% fees, pick one at random and
route their entire $5 transaction through it
* if they're lucky, and a $1000 payment has recently gone the other
way, they succed and pay $0.00005 in tx fees
* if they're not (10% of the time), then both the cheap channels will
be drained, and they'll route their $5 payment through the expensive
channel, suffering both latency while they find a workable path, and
paying much higher fees ($0.02)
ie, some people are paying much higher fees (400x higher than the 90%
of lucky people, or ~10x higher than everyone in the max_msat scenario).
> > I don't think it's ideal if a world that includes both:
> > * altruists, who'll forward your payments for cheap/free
> > * profiteers, who are trying to make a living offering lightning
> > services
> In the world outside of Lightning, we usually make the stink eye at any profiteers who exploit altruists and make noises about how such profiteers are evil, and occasionally even punish them a little bit.
Mmm. You (and CLBOSS) are the profiteer exploiting altruists with all
your aggressive rebalancing ideas in this scenario. Be careful what you
wish for maybe?
In a normally functioning economy, you just find the altruists come up
with some way other than prices to limit who they "sell" too -- whether
that be first come first served, or charities only giving alms to the
obviously needy, or clubs that only provide benefits to members, etc.
You could think of max_msat throttling in exactly that role. But, if it
can be made to work, it's more broadly useful than that, since what it's
fundamentally about is avoiding the "whoops, this channel's depleted,
better retry routing via some other path" cases which applies even in
a world without CLBOSS rebalancing (ie, rationing via the "first come
first served" principle).
> > I mean, that's already a fundamental problem with max_msat: why wouldn't
> > payment initiators do exactly that in the first place?
> Indeed, which suggests that the use of `htlc_max_msat` for a flow valve is flawed:
It was first proposed barely a week ago, of course it's flawed. The
question is what the flaws are, and whether they're fundamental or
things that can be worked around.
> it requires continuous payment flow / payment size curve, and the payment size can be manipulated trivially by any third parties.
The blog post was based on a markov model, which has discrete payment
steps by its nature, so I don't know what you're talking about with a
"continuous" payment flow curve. Both "all payments are at the max size
(simplified as 1 unit)" and "payments are uniform between (0, max)" were
modelled, the former seeming a reasonable match to the "manipulation"
you're proposing, at least for a first brush on the topic.
> > I wonder whether a (per source/dest channel?) token bucket rate limit
> > might suffice, though. [...]
> Given that a published channel is a global resource, any rate limit is going to be shared amongst many users, and if you underquote the value of the IPpvHg you are providing, rebalancers are going to grab as much of the rate limit as they can.
The goal here isn't to stop rebalancing, it's to achieve balanced payment
flow by reducing payment attempts, so that fewer payment attempts
fail. (Rather than achieving balanced payment flows by having many
payment attempts fail)
> I think this implies we need to have a mechanism to move funds outside of the Lightning network, i.e. every published node should have onchain/offchain swap capability.
I guess it would be appealing to a profiteer to have a way of forcing
nodes operating competing channels to taint their wallets by sending to
OFAC banned addresses... Sure seems like the sort of feature that should
be opt-in only to me though.
Cheers,
aj