Mr Hodlen [ARCHIVE] on Nostr: 📅 Original date posted:2018-07-23 📝 Original message: Hi, new guy here. I've ...
📅 Original date posted:2018-07-23
📝 Original message:
Hi, new guy here. I've also been a bit confused about the process from just
reading the BOLTs.
I'm working on an implementation of the BOLTs with the intention of
experimenting with new features.
My concerns are mainly about assignment of values for types and feature bits,
and how we can test new features without conflicting with other developers.
For example, looking at the current BOLT#9, it appears bits 8/9 are the next
free bits available for use, so if I were implementing a new optional feature,
I would pick the next available bits,
however, I find that other developers have also opted to use the same bits:
https://github.com/rgb-org/spec/blob/master/04-lightning-network.md
I think issue of requiring centralized number assignment can be easily avoided
for experimental features by using a unique identifier for the feature instead,
such as a hash of the feature name.
This should make it simpler for independent developers to try new features
before creating new pull requests for a BOLT assignment.
Here's an extremely rough draft of the idea. The intent is that once an
experiment has been tested and is succesful, it can move towards being
standardized as a BOLT. Is it worth adding something like this?
--- BOLT #(-1): Experimental Features [tentatively named]
Overview
The experimental features protocol is intended for testing protocol features
which are not necessarily intended to become standard BOLTs.
* Lightning base protocol messages:
** experiment
A message for arbitrary protocol implementaion with a unique ID for each
feature supported.
type: <??>
data:
[32: experiment_name_hash]
[2: payloadlen]
[payloadlen: payload]
`experiment_name_hash` is be a SHA256 of the feature name, in ASCII,
concatenated with a good source of randomness to avoid any conflict if any two
developers chose the same name.
`payloadlen` is the length of the remaining message, which MUST equal the
length of the outer message -34 bytes.
`payload` is the payload of the experiment message, which should have the same
format as a regular Lightning message described in BOLT #1.
A sending node:
- MUST send the `init_experiments` message with `experiment_name_hash` set
to 0 before any other `experiment` message is sent.
- MUST NOT send an experiment message with any non-zero
`experiment_name_hash` if the receiver has not already declared support for
the feature_hash.
A receiving node:
- If experiment_name_hash is zero:
- Treat the payload as the same format of lightning message, ie, a 16-
bit big endian experiment_type followed by the data.
- If the experiment_name_hash is not recognized or disabled, MUST ignore
the message. (SHOULD also fail the channels?)
Rationale:
The base lightning layer allows extension through feature bits and new odd
message types, but these require centralized assignment by the Lightning
community.
This creates a chicken and egg problem for developers testing experimental
features: In order to be assigned numbers, we need to standarize our feature
through the BOLT process,
but we might not wish to attempt standardization until our experimental
features have been tested.
Without standardization, different teams working on different experimental
features may cause conflicts if different clients are treating feature bits or
message types differently.
By moving all experimental features to a new message where they are wrapped in
a unique feature name, this eradicates chance of conflicting implementations.
Additionally, this message can serve as a generic transport mechanism between
any two lightning nodes who have agreed to support the expierment_name_hash,
as there is no restriction on the format of the payload. This may make it
possible to serve e.g: HTTP over Lightning.
* General experiment messages:
If `experiment_name_hash` in the experiment message is 0, treat its payload as
on of the following messages:
** init_experiments message
Informs a peer of features supported by the client.
1. experiment_type: 16
2. data:
* [2: eflen]
* [eflen*32: experiment_name_hashes]
A sending node:
* MUST send the `init_experiments` message before any other `experiment`
message for each connection.
* SHOULD send the `experiment_name_hash` for any features supported and set
to enabled in their software client.
A receiving node:
* For each experiment_name_hash:
* If the hash is unknown or 0: Ignore the feature
* If the hash is known: SHOULD enable the feature for communication with
this peer.
** experiment_error message
experiment_type: 17
data:
[32: channel_id]
[32: experiment_name_hash]
[2: len]
[len: data]
For all messages before funding_created: Must use temporary_channel_id in lieu
of channel_id.
A sending node:
* If error is critical, should also send the regular lightning `error`
message from BOLT #1
* If the error is not specific to any channel: set channel_id to 0.
A receiving node
* If experiment_name_hash is unknown:
- MUST fail the channel.
* If channel_id is 0
- MUST fail all the channels
Rationale
This message is not intended to replace `error` for criticial errors, but is
provided for additional debugging information
related to the experimental feature being used.
A client may decide whether or not it can recover from such errors
individually per experimental feature, which may include aborting channels and
the connection.
TODO: Define gossip/query messages related to nodes/channels which support
features by experiment_hash_name.
---EOF
On Sunday, 22 July 2018 13:32:02 BST Olaoluwa Osuntokun wrote:
> No need to apologize! Perhaps this confusion shows that the current process
> surrounding creating/modifying/drafting BOLT documents does indeed need to
> be better documented. We've more or less been rolling along with a pretty
> minimal process among the various implementations which I'd say has worked
> pretty well so far. However as more contributors get involved we may need
> to add a bit more structure to ensure things are transparent for newcomers.
>
> On Sun, Jul 22, 2018, 12:57 PM René Pickhardt <r.pickhardt at googlemail.com>
>
> wrote:
> > Sorry did not realized that BOLTs are the equivalent - and aparently many
> > people I spoke to also didn't realize that.
> >
> > I thought BOLT is the protocol specification and the bolts are just the
> > sections. And the BOLT should be updated to a new version.
> >
> > Also I suggested that this should take place for example within the
> > lightning rfc repo. So my suggestion was not about creating another place
> > but more about making the process more transparent or kind of filling the
> > gap that I felt was there.
> >
> > I am sorry for spaming mailboxes with my suggestion just because I didn't
> > understand the current process.
> >
> >
> > Olaoluwa Osuntokun <laolu32 at gmail.com> schrieb am So., 22. Juli 2018
> >
> > 20:59:
> >> We already have the equiv of improvement proposals: BOLTs. Historically
> >> new standardization documents are proposed initially as issues or PR's
> >> when
> >> ultimately accepted. Why do we need another repo?
> >>
> >> On Sun, Jul 22, 2018, 6:45 AM René Pickhardt via Lightning-dev <
> >>
> >> lightning-dev at lists.linuxfoundation.org> wrote:
> >>> Hey everyone,
> >>>
> >>> in the grand tradition of BIPs I propose that we also start to have our
> >>> own LIPs (Lightning Network Improvement proposals)
> >>>
> >>> I think they should be placed on the github.com/lightning account in a
> >>> repo called lips (or within the lightning rfc repo) until that will
> >>> happen
> >>> I created a draft for LIP-0001 (which is describing the process and is
> >>> 95%
> >>> influenced by BIP-0002) in my github repo:
> >>>
> >>> https://github.com/renepickhardt/lips (There are some open Todos and
> >>> Questions in this LIP)
> >>>
> >>> The background for this Idea: I just came home from the bitcoin munich
> >>> meetup where I held a talk examining BOLT. As I was asked to also talk
> >>> about the future plans of the developers for BOLT 1.1 I realized while
> >>> preparing the talk that many ideas are distributed within the community
> >>> but
> >>> it seems we don't have a central place where we collect future
> >>> enhancements
> >>> for BOLT1.1. Having this in mind I think also for the meeting in
> >>> Australia
> >>> it would be nice if already a list of LIPs would be in place so that the
> >>> discussion can be more focused.
> >>> potential LIPs could include:
> >>> * Watchtowers
> >>> * Autopilot
> >>> * AMP
> >>> * Splicing
> >>> * Routing Protcols
> >>> * Broadcasting past Routing statistics
> >>> * eltoo
> >>> * ...
> >>>
> >>> As said before I would volunteer to work on a LIP for Splicing (actually
> >>> I already started)
> >>>
> >>> best Rene
> >>>
> >>>
> >>> --
> >>> https://www.rene-pickhardt.de
> >>>
> >>> Skype: rene.pickhardt
> >>>
> >>> mobile: +49 (0)176 5762 3618
> >>> _______________________________________________
> >>> Lightning-dev mailing list
> >>> Lightning-dev at lists.linuxfoundation.org
> >>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
> >
> > Am 22.07.2018 20:59 schrieb "Olaoluwa Osuntokun" <laolu32 at gmail.com>:
> >
> > We already have the equiv of improvement proposals: BOLTs. Historically
> > new standardization documents are proposed initially as issues or PR's
> > when
> > ultimately accepted. Why do we need another repo?
> >
> > On Sun, Jul 22, 2018, 6:45 AM René Pickhardt via Lightning-dev <
> >
> > lightning-dev at lists.linuxfoundation.org> wrote:
> >> Hey everyone,
> >>
> >> in the grand tradition of BIPs I propose that we also start to have our
> >> own LIPs (Lightning Network Improvement proposals)
> >>
> >> I think they should be placed on the github.com/lightning account in a
> >> repo called lips (or within the lightning rfc repo) until that will
> >> happen
> >> I created a draft for LIP-0001 (which is describing the process and is
> >> 95%
> >> influenced by BIP-0002) in my github repo:
> >>
> >> https://github.com/renepickhardt/lips (There are some open Todos and
> >> Questions in this LIP)
> >>
> >> The background for this Idea: I just came home from the bitcoin munich
> >> meetup where I held a talk examining BOLT. As I was asked to also talk
> >> about the future plans of the developers for BOLT 1.1 I realized while
> >> preparing the talk that many ideas are distributed within the community
> >> but
> >> it seems we don't have a central place where we collect future
> >> enhancements
> >> for BOLT1.1. Having this in mind I think also for the meeting in
> >> Australia
> >> it would be nice if already a list of LIPs would be in place so that the
> >> discussion can be more focused.
> >> potential LIPs could include:
> >> * Watchtowers
> >> * Autopilot
> >> * AMP
> >> * Splicing
> >> * Routing Protcols
> >> * Broadcasting past Routing statistics
> >> * eltoo
> >> * ...
> >>
> >> As said before I would volunteer to work on a LIP for Splicing (actually
> >> I already started)
> >>
> >> best Rene
> >>
> >>
> >> --
> >> https://www.rene-pickhardt.de
> >>
> >> Skype: rene.pickhardt
> >>
> >> mobile: +49 (0)176 5762 3618
> >> _______________________________________________
> >> Lightning-dev mailing list
> >> Lightning-dev at lists.linuxfoundation.org
> >> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
📝 Original message:
Hi, new guy here. I've also been a bit confused about the process from just
reading the BOLTs.
I'm working on an implementation of the BOLTs with the intention of
experimenting with new features.
My concerns are mainly about assignment of values for types and feature bits,
and how we can test new features without conflicting with other developers.
For example, looking at the current BOLT#9, it appears bits 8/9 are the next
free bits available for use, so if I were implementing a new optional feature,
I would pick the next available bits,
however, I find that other developers have also opted to use the same bits:
https://github.com/rgb-org/spec/blob/master/04-lightning-network.md
I think issue of requiring centralized number assignment can be easily avoided
for experimental features by using a unique identifier for the feature instead,
such as a hash of the feature name.
This should make it simpler for independent developers to try new features
before creating new pull requests for a BOLT assignment.
Here's an extremely rough draft of the idea. The intent is that once an
experiment has been tested and is succesful, it can move towards being
standardized as a BOLT. Is it worth adding something like this?
--- BOLT #(-1): Experimental Features [tentatively named]
Overview
The experimental features protocol is intended for testing protocol features
which are not necessarily intended to become standard BOLTs.
* Lightning base protocol messages:
** experiment
A message for arbitrary protocol implementaion with a unique ID for each
feature supported.
type: <??>
data:
[32: experiment_name_hash]
[2: payloadlen]
[payloadlen: payload]
`experiment_name_hash` is be a SHA256 of the feature name, in ASCII,
concatenated with a good source of randomness to avoid any conflict if any two
developers chose the same name.
`payloadlen` is the length of the remaining message, which MUST equal the
length of the outer message -34 bytes.
`payload` is the payload of the experiment message, which should have the same
format as a regular Lightning message described in BOLT #1.
A sending node:
- MUST send the `init_experiments` message with `experiment_name_hash` set
to 0 before any other `experiment` message is sent.
- MUST NOT send an experiment message with any non-zero
`experiment_name_hash` if the receiver has not already declared support for
the feature_hash.
A receiving node:
- If experiment_name_hash is zero:
- Treat the payload as the same format of lightning message, ie, a 16-
bit big endian experiment_type followed by the data.
- If the experiment_name_hash is not recognized or disabled, MUST ignore
the message. (SHOULD also fail the channels?)
Rationale:
The base lightning layer allows extension through feature bits and new odd
message types, but these require centralized assignment by the Lightning
community.
This creates a chicken and egg problem for developers testing experimental
features: In order to be assigned numbers, we need to standarize our feature
through the BOLT process,
but we might not wish to attempt standardization until our experimental
features have been tested.
Without standardization, different teams working on different experimental
features may cause conflicts if different clients are treating feature bits or
message types differently.
By moving all experimental features to a new message where they are wrapped in
a unique feature name, this eradicates chance of conflicting implementations.
Additionally, this message can serve as a generic transport mechanism between
any two lightning nodes who have agreed to support the expierment_name_hash,
as there is no restriction on the format of the payload. This may make it
possible to serve e.g: HTTP over Lightning.
* General experiment messages:
If `experiment_name_hash` in the experiment message is 0, treat its payload as
on of the following messages:
** init_experiments message
Informs a peer of features supported by the client.
1. experiment_type: 16
2. data:
* [2: eflen]
* [eflen*32: experiment_name_hashes]
A sending node:
* MUST send the `init_experiments` message before any other `experiment`
message for each connection.
* SHOULD send the `experiment_name_hash` for any features supported and set
to enabled in their software client.
A receiving node:
* For each experiment_name_hash:
* If the hash is unknown or 0: Ignore the feature
* If the hash is known: SHOULD enable the feature for communication with
this peer.
** experiment_error message
experiment_type: 17
data:
[32: channel_id]
[32: experiment_name_hash]
[2: len]
[len: data]
For all messages before funding_created: Must use temporary_channel_id in lieu
of channel_id.
A sending node:
* If error is critical, should also send the regular lightning `error`
message from BOLT #1
* If the error is not specific to any channel: set channel_id to 0.
A receiving node
* If experiment_name_hash is unknown:
- MUST fail the channel.
* If channel_id is 0
- MUST fail all the channels
Rationale
This message is not intended to replace `error` for criticial errors, but is
provided for additional debugging information
related to the experimental feature being used.
A client may decide whether or not it can recover from such errors
individually per experimental feature, which may include aborting channels and
the connection.
TODO: Define gossip/query messages related to nodes/channels which support
features by experiment_hash_name.
---EOF
On Sunday, 22 July 2018 13:32:02 BST Olaoluwa Osuntokun wrote:
> No need to apologize! Perhaps this confusion shows that the current process
> surrounding creating/modifying/drafting BOLT documents does indeed need to
> be better documented. We've more or less been rolling along with a pretty
> minimal process among the various implementations which I'd say has worked
> pretty well so far. However as more contributors get involved we may need
> to add a bit more structure to ensure things are transparent for newcomers.
>
> On Sun, Jul 22, 2018, 12:57 PM René Pickhardt <r.pickhardt at googlemail.com>
>
> wrote:
> > Sorry did not realized that BOLTs are the equivalent - and aparently many
> > people I spoke to also didn't realize that.
> >
> > I thought BOLT is the protocol specification and the bolts are just the
> > sections. And the BOLT should be updated to a new version.
> >
> > Also I suggested that this should take place for example within the
> > lightning rfc repo. So my suggestion was not about creating another place
> > but more about making the process more transparent or kind of filling the
> > gap that I felt was there.
> >
> > I am sorry for spaming mailboxes with my suggestion just because I didn't
> > understand the current process.
> >
> >
> > Olaoluwa Osuntokun <laolu32 at gmail.com> schrieb am So., 22. Juli 2018
> >
> > 20:59:
> >> We already have the equiv of improvement proposals: BOLTs. Historically
> >> new standardization documents are proposed initially as issues or PR's
> >> when
> >> ultimately accepted. Why do we need another repo?
> >>
> >> On Sun, Jul 22, 2018, 6:45 AM René Pickhardt via Lightning-dev <
> >>
> >> lightning-dev at lists.linuxfoundation.org> wrote:
> >>> Hey everyone,
> >>>
> >>> in the grand tradition of BIPs I propose that we also start to have our
> >>> own LIPs (Lightning Network Improvement proposals)
> >>>
> >>> I think they should be placed on the github.com/lightning account in a
> >>> repo called lips (or within the lightning rfc repo) until that will
> >>> happen
> >>> I created a draft for LIP-0001 (which is describing the process and is
> >>> 95%
> >>> influenced by BIP-0002) in my github repo:
> >>>
> >>> https://github.com/renepickhardt/lips (There are some open Todos and
> >>> Questions in this LIP)
> >>>
> >>> The background for this Idea: I just came home from the bitcoin munich
> >>> meetup where I held a talk examining BOLT. As I was asked to also talk
> >>> about the future plans of the developers for BOLT 1.1 I realized while
> >>> preparing the talk that many ideas are distributed within the community
> >>> but
> >>> it seems we don't have a central place where we collect future
> >>> enhancements
> >>> for BOLT1.1. Having this in mind I think also for the meeting in
> >>> Australia
> >>> it would be nice if already a list of LIPs would be in place so that the
> >>> discussion can be more focused.
> >>> potential LIPs could include:
> >>> * Watchtowers
> >>> * Autopilot
> >>> * AMP
> >>> * Splicing
> >>> * Routing Protcols
> >>> * Broadcasting past Routing statistics
> >>> * eltoo
> >>> * ...
> >>>
> >>> As said before I would volunteer to work on a LIP for Splicing (actually
> >>> I already started)
> >>>
> >>> best Rene
> >>>
> >>>
> >>> --
> >>> https://www.rene-pickhardt.de
> >>>
> >>> Skype: rene.pickhardt
> >>>
> >>> mobile: +49 (0)176 5762 3618
> >>> _______________________________________________
> >>> Lightning-dev mailing list
> >>> Lightning-dev at lists.linuxfoundation.org
> >>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
> >
> > Am 22.07.2018 20:59 schrieb "Olaoluwa Osuntokun" <laolu32 at gmail.com>:
> >
> > We already have the equiv of improvement proposals: BOLTs. Historically
> > new standardization documents are proposed initially as issues or PR's
> > when
> > ultimately accepted. Why do we need another repo?
> >
> > On Sun, Jul 22, 2018, 6:45 AM René Pickhardt via Lightning-dev <
> >
> > lightning-dev at lists.linuxfoundation.org> wrote:
> >> Hey everyone,
> >>
> >> in the grand tradition of BIPs I propose that we also start to have our
> >> own LIPs (Lightning Network Improvement proposals)
> >>
> >> I think they should be placed on the github.com/lightning account in a
> >> repo called lips (or within the lightning rfc repo) until that will
> >> happen
> >> I created a draft for LIP-0001 (which is describing the process and is
> >> 95%
> >> influenced by BIP-0002) in my github repo:
> >>
> >> https://github.com/renepickhardt/lips (There are some open Todos and
> >> Questions in this LIP)
> >>
> >> The background for this Idea: I just came home from the bitcoin munich
> >> meetup where I held a talk examining BOLT. As I was asked to also talk
> >> about the future plans of the developers for BOLT 1.1 I realized while
> >> preparing the talk that many ideas are distributed within the community
> >> but
> >> it seems we don't have a central place where we collect future
> >> enhancements
> >> for BOLT1.1. Having this in mind I think also for the meeting in
> >> Australia
> >> it would be nice if already a list of LIPs would be in place so that the
> >> discussion can be more focused.
> >> potential LIPs could include:
> >> * Watchtowers
> >> * Autopilot
> >> * AMP
> >> * Splicing
> >> * Routing Protcols
> >> * Broadcasting past Routing statistics
> >> * eltoo
> >> * ...
> >>
> >> As said before I would volunteer to work on a LIP for Splicing (actually
> >> I already started)
> >>
> >> best Rene
> >>
> >>
> >> --
> >> https://www.rene-pickhardt.de
> >>
> >> Skype: rene.pickhardt
> >>
> >> mobile: +49 (0)176 5762 3618
> >> _______________________________________________
> >> Lightning-dev mailing list
> >> Lightning-dev at lists.linuxfoundation.org
> >> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev