Tony Giorgio [ARCHIVE] on Nostr: đź“… Original date posted:2022-06-08 đź“ť Original message: Hi /dev/fd0, > Can this ...
đź“… Original date posted:2022-06-08
đź“ť Original message:
Hi /dev/fd0,
> Can this be fixed by making error messages less verbose or reveal less information?
​
Not completely fixed, but much more time consuming to exploit. When a prober gets _just_ the SCID correct, a certain error is passed back. Typically incorrect_cltv_expiry​ or amount_below_minimum​ being the error, proving that the channel ID is correct. Then it's just up to getting the other values correct. So if the error messages were generic and did not reveal the Channel ID was correct unless all of the other values were also correct, it requires many more combinations of parameters to probe for, instead of just the Channel ID, then just the CLTV, then the amount, then the other node. Does not fix the problem completely because eventually if you were to get all values correct, it would route to the other node, where the other node creates the incorrect_or_unknown_payment_details​ error, which you can tell which node creates the error, proving it got to the guessed node.
> Are alias SCIDs necessary to fix it or error messages alone can fix it?
​
With the above said, I believe it is necessary to move to SCIDs and that error messages alone can't fix it completely. It's better anyways because when an invoice is made with routing hints for unannounced channels, those consuming the invoice can't associate UTXO's with the node creating the invoice if aliases were used.
Also, in addition to LDK implementing it, I know LND is working on it currently too https://github.com/lightningnetwork/lnd/pull/5955
> I love the research and thanks for sharing all the information. I am assuming analytic firms would be using this already.
​
Thank you! I assume it as well though it's probably easy to tell whether this is happening to your node or not if someone was watching their HTLC routing failure reasons.
Thanks,
Tony
------- Original Message -------
On Tuesday, June 7th, 2022 at 10:36 PM, alicexbt <alicexbt at protonmail.com> wrote:
> Hi Tony,
>
>> The reason this is possible is because probing is a free operation on the Lightning Network after a channel is opened, the error reasons given are way too verbose, and currently channel IDs are based on UTXO's. Scid aliases may be the biggest benefit here, but the use of `unknown_next_peer` , `invalid_onion_hmac`, `incorrect_cltv_expiry`, and `amount_below_minimum` have been the biggest helpers in exploiting channel privacy.
>
> Can this be fixed by making error messages less verbose or reveal less information?
>
>> We should definitely migrate to alias scid's, and encourage every active unannounced channel holder to close, coinjoin, and reopen with an alias. But care should be given in the future when it comes to error reasons revealing information that is meant to be "private". Until this migration happens, it would be beneficial to stop being so specific about errors, this does not really seem to help end users anyways.
>
> Alias SCID would be better for privacy as they allow a node to request a channel by a random value instead of value derived from the on-chain transaction. Are alias SCIDs necessary to fix it or error messages alone can fix it?
>
> I found these pull requests and assuming alias SCID are already implemented in LDK/rust-lightning:
>
> [https://github.com/lightningdevkit/rust-lightning/pull/1311/](https://github.com/lightningdevkit/rust-lightning/pull/1311/commits)
> https://github.com/lightningdevkit/rust-lightning/pull/1351
>
>> I'll be continuing with this probing project while the problem exists, and work on narrowing down the other channel partner and fixing efficiency bugs. I am publicizing the results as I go, so fair warning that if you have any unannounced channels that you assumed were private and need them to be, close them now on the off chance they get revealed. This could have always been happening already already by analytic firms, so I hope by publicizing this we are all on the same playing field. It is also beneficial to get a better estimate of the unknown size of the Lightning Network.
>
> I love the research and thanks for sharing all the information. I am assuming analytic firms would be using this already.
>
> /dev/fd0
>
> Sent with [Proton Mail](https://proton.me/) secure email.
>
> ------- Original Message -------
> On Wednesday, June 8th, 2022 at 7:35 AM, Tony Giorgio via Lightning-dev <lightning-dev at lists.linuxfoundation.org> wrote:
>
>> Hi,
>>
>> For the past few months I have been working on an LDK probing project that searches for unannounced channels on the Lightning Network. For the past week, I have been probing on mainnet and squashing bugs / making optimizations.
>>
>> So far I have found near 445 unannounced channels totaling 1,076,077,750 satoshi's locked across the 3 nodes I have probed, some with just a minimized set (~30,000) of probable channels based on "round payment amount" and "1 or 2 tx output" heuristics on P2WSH UTXO's. Most of them being on Aincq's node found with the minimized set, I've yet to run the complete set with them. There are about ~860,000 P2WSH UTXO's, about ~60,000 of which are public, so the upward limit of possible private channels is around ~800,000.
>>
>> The exact results are publicized here: https://github.com/BitcoinDevShop/hidden-lightning-network/blob/master/data/results/results.json
>>
>> The reason this is possible is because probing is a free operation on the Lightning Network after a channel is opened, the error reasons given are way too verbose, and currently channel IDs are based on UTXO's. Scid aliases may be the biggest benefit here, but the use of `unknown_next_peer` , `invalid_onion_hmac`, `incorrect_cltv_expiry`, and `amount_below_minimum` have been the biggest helpers in exploiting channel privacy.
>>
>> By creating a probe guessing the Channel ID based on unspent p2wsh transactions, it's a `m * n` problem to probe the entire network, where `m` is utxos and `n` is nodes. Without these errors and instead something like `temporary_channel_failure` or a generic indistinguishable error, guessing a Channel ID would come down to an upwards of `m * n * n-1 * ~2000`, which would be each utxo with each pairing of nodes, each with about ~2000 cltv's to guess (numbers are as low as 7 to as high as ~2000). I threw the extra 2000 into the equation because even with `800,000 * 1 * 2000`, it gets much more time consuming to even probe a single node when we're already spending upwards of a day or so for near 1 million or 2 probes. Concurrent probing is possible, but starts to require more locked up liquidity.
>>
>> We should definitely migrate to alias scid's, and encourage every active unannounced channel holder to close, coinjoin, and reopen with an alias. But care should be given in the future when it comes to error reasons revealing information that is meant to be "private". Until this migration happens, it would be beneficial to stop being so specific about errors, this does not really seem to help end users anyways.
>>
>> I'll be continuing with this probing project while the problem exists, and work on narrowing down the other channel partner and fixing efficiency bugs. I am publicizing the results as I go, so fair warning that if you have any unannounced channels that you assumed were private and need them to be, close them now on the off chance they get revealed. This could have always been happening already already by analytic firms, so I hope by publicizing this we are all on the same playing field. It is also beneficial to get a better estimate of the unknown size of the Lightning Network.
>>
>> For more about this project and viewing the dataset, go to http://hiddenlightningnetwork.com
>>
>> Thanks,
>> Tony
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220608/7760d9dd/attachment-0001.html>;
đź“ť Original message:
Hi /dev/fd0,
> Can this be fixed by making error messages less verbose or reveal less information?
​
Not completely fixed, but much more time consuming to exploit. When a prober gets _just_ the SCID correct, a certain error is passed back. Typically incorrect_cltv_expiry​ or amount_below_minimum​ being the error, proving that the channel ID is correct. Then it's just up to getting the other values correct. So if the error messages were generic and did not reveal the Channel ID was correct unless all of the other values were also correct, it requires many more combinations of parameters to probe for, instead of just the Channel ID, then just the CLTV, then the amount, then the other node. Does not fix the problem completely because eventually if you were to get all values correct, it would route to the other node, where the other node creates the incorrect_or_unknown_payment_details​ error, which you can tell which node creates the error, proving it got to the guessed node.
> Are alias SCIDs necessary to fix it or error messages alone can fix it?
​
With the above said, I believe it is necessary to move to SCIDs and that error messages alone can't fix it completely. It's better anyways because when an invoice is made with routing hints for unannounced channels, those consuming the invoice can't associate UTXO's with the node creating the invoice if aliases were used.
Also, in addition to LDK implementing it, I know LND is working on it currently too https://github.com/lightningnetwork/lnd/pull/5955
> I love the research and thanks for sharing all the information. I am assuming analytic firms would be using this already.
​
Thank you! I assume it as well though it's probably easy to tell whether this is happening to your node or not if someone was watching their HTLC routing failure reasons.
Thanks,
Tony
------- Original Message -------
On Tuesday, June 7th, 2022 at 10:36 PM, alicexbt <alicexbt at protonmail.com> wrote:
> Hi Tony,
>
>> The reason this is possible is because probing is a free operation on the Lightning Network after a channel is opened, the error reasons given are way too verbose, and currently channel IDs are based on UTXO's. Scid aliases may be the biggest benefit here, but the use of `unknown_next_peer` , `invalid_onion_hmac`, `incorrect_cltv_expiry`, and `amount_below_minimum` have been the biggest helpers in exploiting channel privacy.
>
> Can this be fixed by making error messages less verbose or reveal less information?
>
>> We should definitely migrate to alias scid's, and encourage every active unannounced channel holder to close, coinjoin, and reopen with an alias. But care should be given in the future when it comes to error reasons revealing information that is meant to be "private". Until this migration happens, it would be beneficial to stop being so specific about errors, this does not really seem to help end users anyways.
>
> Alias SCID would be better for privacy as they allow a node to request a channel by a random value instead of value derived from the on-chain transaction. Are alias SCIDs necessary to fix it or error messages alone can fix it?
>
> I found these pull requests and assuming alias SCID are already implemented in LDK/rust-lightning:
>
> [https://github.com/lightningdevkit/rust-lightning/pull/1311/](https://github.com/lightningdevkit/rust-lightning/pull/1311/commits)
> https://github.com/lightningdevkit/rust-lightning/pull/1351
>
>> I'll be continuing with this probing project while the problem exists, and work on narrowing down the other channel partner and fixing efficiency bugs. I am publicizing the results as I go, so fair warning that if you have any unannounced channels that you assumed were private and need them to be, close them now on the off chance they get revealed. This could have always been happening already already by analytic firms, so I hope by publicizing this we are all on the same playing field. It is also beneficial to get a better estimate of the unknown size of the Lightning Network.
>
> I love the research and thanks for sharing all the information. I am assuming analytic firms would be using this already.
>
> /dev/fd0
>
> Sent with [Proton Mail](https://proton.me/) secure email.
>
> ------- Original Message -------
> On Wednesday, June 8th, 2022 at 7:35 AM, Tony Giorgio via Lightning-dev <lightning-dev at lists.linuxfoundation.org> wrote:
>
>> Hi,
>>
>> For the past few months I have been working on an LDK probing project that searches for unannounced channels on the Lightning Network. For the past week, I have been probing on mainnet and squashing bugs / making optimizations.
>>
>> So far I have found near 445 unannounced channels totaling 1,076,077,750 satoshi's locked across the 3 nodes I have probed, some with just a minimized set (~30,000) of probable channels based on "round payment amount" and "1 or 2 tx output" heuristics on P2WSH UTXO's. Most of them being on Aincq's node found with the minimized set, I've yet to run the complete set with them. There are about ~860,000 P2WSH UTXO's, about ~60,000 of which are public, so the upward limit of possible private channels is around ~800,000.
>>
>> The exact results are publicized here: https://github.com/BitcoinDevShop/hidden-lightning-network/blob/master/data/results/results.json
>>
>> The reason this is possible is because probing is a free operation on the Lightning Network after a channel is opened, the error reasons given are way too verbose, and currently channel IDs are based on UTXO's. Scid aliases may be the biggest benefit here, but the use of `unknown_next_peer` , `invalid_onion_hmac`, `incorrect_cltv_expiry`, and `amount_below_minimum` have been the biggest helpers in exploiting channel privacy.
>>
>> By creating a probe guessing the Channel ID based on unspent p2wsh transactions, it's a `m * n` problem to probe the entire network, where `m` is utxos and `n` is nodes. Without these errors and instead something like `temporary_channel_failure` or a generic indistinguishable error, guessing a Channel ID would come down to an upwards of `m * n * n-1 * ~2000`, which would be each utxo with each pairing of nodes, each with about ~2000 cltv's to guess (numbers are as low as 7 to as high as ~2000). I threw the extra 2000 into the equation because even with `800,000 * 1 * 2000`, it gets much more time consuming to even probe a single node when we're already spending upwards of a day or so for near 1 million or 2 probes. Concurrent probing is possible, but starts to require more locked up liquidity.
>>
>> We should definitely migrate to alias scid's, and encourage every active unannounced channel holder to close, coinjoin, and reopen with an alias. But care should be given in the future when it comes to error reasons revealing information that is meant to be "private". Until this migration happens, it would be beneficial to stop being so specific about errors, this does not really seem to help end users anyways.
>>
>> I'll be continuing with this probing project while the problem exists, and work on narrowing down the other channel partner and fixing efficiency bugs. I am publicizing the results as I go, so fair warning that if you have any unannounced channels that you assumed were private and need them to be, close them now on the off chance they get revealed. This could have always been happening already already by analytic firms, so I hope by publicizing this we are all on the same playing field. It is also beneficial to get a better estimate of the unknown size of the Lightning Network.
>>
>> For more about this project and viewing the dataset, go to http://hiddenlightningnetwork.com
>>
>> Thanks,
>> Tony
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxfoundation.org/pipermail/lightning-dev/attachments/20220608/7760d9dd/attachment-0001.html>;