Olaoluwa Osuntokun [ARCHIVE] on Nostr: 📅 Original date posted:2022-03-22 📝 Original message: Hi Rusty, > AMP seems to ...
📅 Original date posted:2022-03-22
📝 Original message:
Hi Rusty,
> AMP seems to be a Lightning Labs proprietary extension. You mean keysend,
> which at least has a draft spec?
I'm confused: the keyspend specification lives today as a very succinct bLIP
[1] (use this record that contains the preimage settle the payment), and the
AMP spec has had an open PR [2] (which admittedly needs to be mega-rebased)
for sometime now and is amongst the oldest PRs in the BOLT tracker. Your
definition of a "proprietary extension" here would seem to cover just about
every open BOLT pull request that hasn't yet been widely adopted by multiple
implementations.
With that aside, AMP is used here instead of keysend as each payment split
uses a different payment hash, which makes it harder to correlate payment
splits as both the amount and hash will differ. Also given that the e2e
payment needs to drop off additional fees (to pay for the forwarding pass)
to each hop, the receiver doesn't actually know how much to expect as total
payment (no invoice exists in this scenario). With keysend, it's possible
the receiver accidentally pulls early, with the funds irrevocably sent,
while no true onion messaging session has been constructed. With AMP the
sender is the one that controls _when_ all the funds can be pulled, which
eliminates such edge cases.
> Sure, let's keep encouraging people to use HTLCs for free to send data? I
> can certainly implement that if you prefer!
I think the difference here is that HTLCs aren't that great for sending
data: any space you allocate to arbitrary data takes away from the total
route length. Also given the existence of the max_htlc parameters in channel
updates, nodes can increase this value which allows them to discourage
certain classes of behavior, as the HTLC that transmits the data serves
effectively as an ephemeral bond, meaning that HTLC data transfer attempts
incur an opportunity cost w.r.t the capital locked up.
As a result, what's deployed in the wild today that uses HTLCs to send data
(importantly at no additional marginal cost, since you sling 1.3KB across
the wire anyway for normal payments), imo is mostly cases that only need a
smaller amount of data, or use it mainly as a signalling layer. In contrast,
onion messaging would allow users to send 32 KB (and beyond?), which makes
certain classes of applications more feasible.
> I suggest naively limiting to 10/sec for peers with channels, and 1/sec
> for peers without for now.
Curious to see what a fleshed out version of this would look like. IMO if
this rate limiting event doesn't somehow make it back to the sender, then it
isn't clear exactly why they're unable to even fetch an invoice to _attempt_
a payment. I think one issue with defaulting to always fetching invoices in
a multi-hop manner is that a node failure may mean that a user can't even
obtain an invoice, compared to just hitting an API or scanning a QR code
that encodes one (pretty much 99% chance of success).
(I think there's also a side discussion here w.r.t if tightly coupling an
invoice/offer to a single node is compatible with the multi-node receive
architectures that some services are beginning to develop/deploy, but we can
make another thread for that.)
> You mean:
> 1. Don't use an even TLV field.
I don't see why one _wouldn't_ use an even field here: if the node doesn't
know what this means, then I want them to reject the payment, as otherwise
they aren't able to properly do the bandwidth metering. Even assuming a
feature bit, I think an even field would make sense, but ofc I'm open to
further reasoning.
> I think it's a premature optimization? Make standard duration 2016
> blocks; then they can request multiples if they want? Reduces
> node_announcement size.
Hmm, potentially but if I just need this to ask a node about their latest
ephemeral onion service, do I really need the session to persist for 2016
blocks? At just 8 bytes here, it's smaller than a tor v3 onion address and
even most feature bit distributions. Also there's nothing stopping someone
from advertising optional feature bit 12,324 which means that nodes need to
lug around and store all those extra bytes.
> I'd be tempted to use 16 bytes? Collisions here are not really a thing
> since you'd need a network packet per probe, and you're time limited.
Yeah if the 32 bytes for each hop ends up eating too much into the total
route length, then most use cases can get by with 16 bytes.
> AFAICT this is easy to implement on top of onion_messages as they stand
> today (if you don't want to fwd free onion messages at all, don't set
> that bit?).
Correct! The idea is to give nodes that don't want to forward for free an
option to charge for forwarding and supplement their existing routing node
revenue.
On Twitter, Lisa brought up that as the data forwarding and payment aren't
"atomic", you're reliant on the nodes to actually "do the thing" once the
payment has been completed. This is correct, and imo can be mitigated mainly
via a "tit for tat approach", so paying for a smol session and seeing if
they actually cooperate. Incentive wise, if the node continues to operate as
expected, then they can expect more messaging sessions to be created, which
means mo sats as revenue. I think we're seeing a similar dynamic play out to
day in the network, as path finding algorithms continue to evolve to factor
in "reliability" information (usually some derived probability of success),
which can tend to favor well managed and responsive nodes.
[1]: https://github.com/lightning/blips/blob/master/blip-0003.md
[2]: https://github.com/lightning/bolts/pull/658
-- Laolu
On Wed, Feb 23, 2022 at 8:37 PM Rusty Russell <rusty at rustcorp.com.au> wrote:
> Olaoluwa Osuntokun <laolu32 at gmail.com> writes:
> > Hi y'all,
> >
> > (TL;DR: a way to nodes to get paid to forward onion messages by adding an
> > upfront session creation phase that uses AMP tender a messaging session
> to a
> > receiver, with nodes being paid upfront for purchase of forwarding
> > bandwidth, and a session identifier being transmitted alongside onion
> > messages to identify paid sessions)
>
> AMP seems to be a Lightning Labs proprietary extension. You mean
> keysend, which at least has a draft spec?
>
> > Onion messaging has been proposed as a way to do things like fetch
> invoices
> > directly from a potential receiver _directly_ over the existing LN. The
> > current proposal (packaged under the BOLT 12 umbrella) uses a new message
> > (`onion_message`) that inherits the design of the existing Sphinx-based
> > onion blob included in htlc_add messages as a way to propagate arbitrary
> > messages across the network. Blinded paths which are effectively an
> unrolled
> > Sphinx SURB (single use reply block), are used to support reply messages
> in
> > a more private manner. Compared to SURBs, blinded paths are more
> flexible as
> > they don't lock in things like fees or CLTV values.
> >
> > A direct outcome of widespread adoption of the proposal is that the
> scope of
> > LN is expanded beyond "just" a decentralized p2p payment system, with the
>
> Sure, let's keep encouraging people to use HTLCs for free to send data?
> I can certainly implement that if you prefer!
>
> > 1. As there's no explicit session creation/acceptance, a node can be
> > spammed with unsolicited messages with no way to deny unwanted messages
> nor
> > explicitly allow messages from certain senders.
> >
> > 2. Nodes that forward these messages (up to 32 KB per message) receive
> no
> > compensation for the network bandwidth their expend, effectively
> shuffling
> > around messages for free.
> >
> > 3. Rate limiting isn't concretely addressed, which may result in
> > heterogeneous rate limiting policies enforced around the network, which
> can
> > degrade the developer/user experience (why are my packets being randomly
> > dropped?).
>
> Sure, this is a fun one! I can post separately on ratelimiting; I
> suggest naively limiting to 10/sec for peers with channels, and 1/sec
> for peers without for now.
>
> (In practice, spamming with HTLCs is infinitely more satisfying...)
>
> > In this email I propose a way to address the issues mentioned above by
> > adding explicit onion messaging session creation as well as a way for
> nodes
> > to be (optionally) paid for any onion messages they forward. In short, an
> > explicit session creation phase is introduced, with the receiver being
> able
> > to accept/deny the session. If the session is accepted, then all nodes
> that
> > comprise the session route are compensated for allotting a certain
> amount of
> > bandwidth to the session (which is ephemeral by nature).
>
> It's an interesting layer on top (esp if you want to stream movies), but
> I never proposed this because it seems to add a source-identifying
> session id, which is a huge privacy step backwards.
>
> You really *do not want* to use this for independent transmissions.
>
> I flirted with using blinded tokens, but it gets complex fast; ideas
> welcome!
>
> > ## Node Announcement TLV Extension
> >
> > In order to allow nodes to signal that they want to be paid to forward
> onion
> > messages and also specify their pricing, we add two new TLV to the
> node_ann
> > message:
> >
> > * type: 1 (`sats_per_byte`)
> > * data:
> > * [`uint64`:`forwarding_rate`]
> > * type: 2 (`sats_per_block`)
> > * data:
> > * [`uint64`:`per_block_rate`]
>
> You mean:
>
> * type: 1 (`sats_per_byte`)
> * data:
> * [`tu64`:`forwarding_rate`]
> * type: 3 (`sats_per_block`)
> * data:
> * [`tu64`:`per_block_rate`]
>
> 1. Don't use an even TLV field.
> 2. Might as well use truncated u64.
>
> > The `sats_per_byte` field allows nodes to price their bandwidth, ensuring
> > that they get paid for each chunk of allocated bandwidth. As sessions
> have a
> > fixed time frame and nodes need to store additional data within that time
> > frame, the `sats_per_block` allows nodes to price this cost, as they'll
> hold
> > onto the session identifier information until the specified block height
> > (detailed below).
> >
> > As onion messages will _typically_ be fixed sized we may want to use
> > coursers
> > metering here instead of bytes, possibly paying for 1.3KB or 32 KB chunks
> > instead.
>
> I think it's a premature optimization? Make standard duration 2016
> blocks; then they can request multiples if they want? Reduces
> node_announcement size.
>
> > With the above nodes are able to express that they're willing to forward
> > messages for sats, and how much they charge per byte as well as per
> block.
> > Next we add a new TLV in the _existing_ HTLC onion blob that allows a
> > sending node to tender paid onion message session creation. A sketch of
> this
> > type would look something like:
> >
> > * type: 14 (`onion_session_id`)
> > * data:
> > * [`32*byte`:`session_id`]
>
> I'd be tempted to use 16 bytes? Collisions here are not really a thing
> since you'd need a network packet per probe, and you're time limited.
>
> > After session creation succeeds, nodes will forward onion messages that
> > include that `onion_session_id`. The set of `encrypted_data_tlv` for
> onion
> > messages is extended to also specify a new type that stores the session
> ID:
> >
> > * type: 10 (`onion_session_id`)
> > * data:
> > * [`32*byte`:`session_id`]
> >
> > When forwarding an onion message that includes an `onion_session_id` (a
> node
> > may only forward messages that contain such an ID), nodes do the
> necessary
> > bookkeeping to tally how much bandwidth if left in this session, and also
> > check that the session hasn't expired before forwarding.
>
> This is good because it doesn't require a db write (if you crash and
> forget to charge, it's OK).
>
> AFAICT this is easy to implement on top of onion_messages as they stand
> today (if you don't want to fwd free onion messages at all, don't set
> that bit?).
>
> Cheers,
> Rusty.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220322/fc6fcf6b/attachment-0001.html>;
📝 Original message:
Hi Rusty,
> AMP seems to be a Lightning Labs proprietary extension. You mean keysend,
> which at least has a draft spec?
I'm confused: the keyspend specification lives today as a very succinct bLIP
[1] (use this record that contains the preimage settle the payment), and the
AMP spec has had an open PR [2] (which admittedly needs to be mega-rebased)
for sometime now and is amongst the oldest PRs in the BOLT tracker. Your
definition of a "proprietary extension" here would seem to cover just about
every open BOLT pull request that hasn't yet been widely adopted by multiple
implementations.
With that aside, AMP is used here instead of keysend as each payment split
uses a different payment hash, which makes it harder to correlate payment
splits as both the amount and hash will differ. Also given that the e2e
payment needs to drop off additional fees (to pay for the forwarding pass)
to each hop, the receiver doesn't actually know how much to expect as total
payment (no invoice exists in this scenario). With keysend, it's possible
the receiver accidentally pulls early, with the funds irrevocably sent,
while no true onion messaging session has been constructed. With AMP the
sender is the one that controls _when_ all the funds can be pulled, which
eliminates such edge cases.
> Sure, let's keep encouraging people to use HTLCs for free to send data? I
> can certainly implement that if you prefer!
I think the difference here is that HTLCs aren't that great for sending
data: any space you allocate to arbitrary data takes away from the total
route length. Also given the existence of the max_htlc parameters in channel
updates, nodes can increase this value which allows them to discourage
certain classes of behavior, as the HTLC that transmits the data serves
effectively as an ephemeral bond, meaning that HTLC data transfer attempts
incur an opportunity cost w.r.t the capital locked up.
As a result, what's deployed in the wild today that uses HTLCs to send data
(importantly at no additional marginal cost, since you sling 1.3KB across
the wire anyway for normal payments), imo is mostly cases that only need a
smaller amount of data, or use it mainly as a signalling layer. In contrast,
onion messaging would allow users to send 32 KB (and beyond?), which makes
certain classes of applications more feasible.
> I suggest naively limiting to 10/sec for peers with channels, and 1/sec
> for peers without for now.
Curious to see what a fleshed out version of this would look like. IMO if
this rate limiting event doesn't somehow make it back to the sender, then it
isn't clear exactly why they're unable to even fetch an invoice to _attempt_
a payment. I think one issue with defaulting to always fetching invoices in
a multi-hop manner is that a node failure may mean that a user can't even
obtain an invoice, compared to just hitting an API or scanning a QR code
that encodes one (pretty much 99% chance of success).
(I think there's also a side discussion here w.r.t if tightly coupling an
invoice/offer to a single node is compatible with the multi-node receive
architectures that some services are beginning to develop/deploy, but we can
make another thread for that.)
> You mean:
> 1. Don't use an even TLV field.
I don't see why one _wouldn't_ use an even field here: if the node doesn't
know what this means, then I want them to reject the payment, as otherwise
they aren't able to properly do the bandwidth metering. Even assuming a
feature bit, I think an even field would make sense, but ofc I'm open to
further reasoning.
> I think it's a premature optimization? Make standard duration 2016
> blocks; then they can request multiples if they want? Reduces
> node_announcement size.
Hmm, potentially but if I just need this to ask a node about their latest
ephemeral onion service, do I really need the session to persist for 2016
blocks? At just 8 bytes here, it's smaller than a tor v3 onion address and
even most feature bit distributions. Also there's nothing stopping someone
from advertising optional feature bit 12,324 which means that nodes need to
lug around and store all those extra bytes.
> I'd be tempted to use 16 bytes? Collisions here are not really a thing
> since you'd need a network packet per probe, and you're time limited.
Yeah if the 32 bytes for each hop ends up eating too much into the total
route length, then most use cases can get by with 16 bytes.
> AFAICT this is easy to implement on top of onion_messages as they stand
> today (if you don't want to fwd free onion messages at all, don't set
> that bit?).
Correct! The idea is to give nodes that don't want to forward for free an
option to charge for forwarding and supplement their existing routing node
revenue.
On Twitter, Lisa brought up that as the data forwarding and payment aren't
"atomic", you're reliant on the nodes to actually "do the thing" once the
payment has been completed. This is correct, and imo can be mitigated mainly
via a "tit for tat approach", so paying for a smol session and seeing if
they actually cooperate. Incentive wise, if the node continues to operate as
expected, then they can expect more messaging sessions to be created, which
means mo sats as revenue. I think we're seeing a similar dynamic play out to
day in the network, as path finding algorithms continue to evolve to factor
in "reliability" information (usually some derived probability of success),
which can tend to favor well managed and responsive nodes.
[1]: https://github.com/lightning/blips/blob/master/blip-0003.md
[2]: https://github.com/lightning/bolts/pull/658
-- Laolu
On Wed, Feb 23, 2022 at 8:37 PM Rusty Russell <rusty at rustcorp.com.au> wrote:
> Olaoluwa Osuntokun <laolu32 at gmail.com> writes:
> > Hi y'all,
> >
> > (TL;DR: a way to nodes to get paid to forward onion messages by adding an
> > upfront session creation phase that uses AMP tender a messaging session
> to a
> > receiver, with nodes being paid upfront for purchase of forwarding
> > bandwidth, and a session identifier being transmitted alongside onion
> > messages to identify paid sessions)
>
> AMP seems to be a Lightning Labs proprietary extension. You mean
> keysend, which at least has a draft spec?
>
> > Onion messaging has been proposed as a way to do things like fetch
> invoices
> > directly from a potential receiver _directly_ over the existing LN. The
> > current proposal (packaged under the BOLT 12 umbrella) uses a new message
> > (`onion_message`) that inherits the design of the existing Sphinx-based
> > onion blob included in htlc_add messages as a way to propagate arbitrary
> > messages across the network. Blinded paths which are effectively an
> unrolled
> > Sphinx SURB (single use reply block), are used to support reply messages
> in
> > a more private manner. Compared to SURBs, blinded paths are more
> flexible as
> > they don't lock in things like fees or CLTV values.
> >
> > A direct outcome of widespread adoption of the proposal is that the
> scope of
> > LN is expanded beyond "just" a decentralized p2p payment system, with the
>
> Sure, let's keep encouraging people to use HTLCs for free to send data?
> I can certainly implement that if you prefer!
>
> > 1. As there's no explicit session creation/acceptance, a node can be
> > spammed with unsolicited messages with no way to deny unwanted messages
> nor
> > explicitly allow messages from certain senders.
> >
> > 2. Nodes that forward these messages (up to 32 KB per message) receive
> no
> > compensation for the network bandwidth their expend, effectively
> shuffling
> > around messages for free.
> >
> > 3. Rate limiting isn't concretely addressed, which may result in
> > heterogeneous rate limiting policies enforced around the network, which
> can
> > degrade the developer/user experience (why are my packets being randomly
> > dropped?).
>
> Sure, this is a fun one! I can post separately on ratelimiting; I
> suggest naively limiting to 10/sec for peers with channels, and 1/sec
> for peers without for now.
>
> (In practice, spamming with HTLCs is infinitely more satisfying...)
>
> > In this email I propose a way to address the issues mentioned above by
> > adding explicit onion messaging session creation as well as a way for
> nodes
> > to be (optionally) paid for any onion messages they forward. In short, an
> > explicit session creation phase is introduced, with the receiver being
> able
> > to accept/deny the session. If the session is accepted, then all nodes
> that
> > comprise the session route are compensated for allotting a certain
> amount of
> > bandwidth to the session (which is ephemeral by nature).
>
> It's an interesting layer on top (esp if you want to stream movies), but
> I never proposed this because it seems to add a source-identifying
> session id, which is a huge privacy step backwards.
>
> You really *do not want* to use this for independent transmissions.
>
> I flirted with using blinded tokens, but it gets complex fast; ideas
> welcome!
>
> > ## Node Announcement TLV Extension
> >
> > In order to allow nodes to signal that they want to be paid to forward
> onion
> > messages and also specify their pricing, we add two new TLV to the
> node_ann
> > message:
> >
> > * type: 1 (`sats_per_byte`)
> > * data:
> > * [`uint64`:`forwarding_rate`]
> > * type: 2 (`sats_per_block`)
> > * data:
> > * [`uint64`:`per_block_rate`]
>
> You mean:
>
> * type: 1 (`sats_per_byte`)
> * data:
> * [`tu64`:`forwarding_rate`]
> * type: 3 (`sats_per_block`)
> * data:
> * [`tu64`:`per_block_rate`]
>
> 1. Don't use an even TLV field.
> 2. Might as well use truncated u64.
>
> > The `sats_per_byte` field allows nodes to price their bandwidth, ensuring
> > that they get paid for each chunk of allocated bandwidth. As sessions
> have a
> > fixed time frame and nodes need to store additional data within that time
> > frame, the `sats_per_block` allows nodes to price this cost, as they'll
> hold
> > onto the session identifier information until the specified block height
> > (detailed below).
> >
> > As onion messages will _typically_ be fixed sized we may want to use
> > coursers
> > metering here instead of bytes, possibly paying for 1.3KB or 32 KB chunks
> > instead.
>
> I think it's a premature optimization? Make standard duration 2016
> blocks; then they can request multiples if they want? Reduces
> node_announcement size.
>
> > With the above nodes are able to express that they're willing to forward
> > messages for sats, and how much they charge per byte as well as per
> block.
> > Next we add a new TLV in the _existing_ HTLC onion blob that allows a
> > sending node to tender paid onion message session creation. A sketch of
> this
> > type would look something like:
> >
> > * type: 14 (`onion_session_id`)
> > * data:
> > * [`32*byte`:`session_id`]
>
> I'd be tempted to use 16 bytes? Collisions here are not really a thing
> since you'd need a network packet per probe, and you're time limited.
>
> > After session creation succeeds, nodes will forward onion messages that
> > include that `onion_session_id`. The set of `encrypted_data_tlv` for
> onion
> > messages is extended to also specify a new type that stores the session
> ID:
> >
> > * type: 10 (`onion_session_id`)
> > * data:
> > * [`32*byte`:`session_id`]
> >
> > When forwarding an onion message that includes an `onion_session_id` (a
> node
> > may only forward messages that contain such an ID), nodes do the
> necessary
> > bookkeeping to tally how much bandwidth if left in this session, and also
> > check that the session hasn't expired before forwarding.
>
> This is good because it doesn't require a db write (if you crash and
> forget to charge, it's OK).
>
> AFAICT this is easy to implement on top of onion_messages as they stand
> today (if you don't want to fwd free onion messages at all, don't set
> that bit?).
>
> Cheers,
> Rusty.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220322/fc6fcf6b/attachment-0001.html>;