mewj on Nostr: My technical analysis of NIP-95 and file distribution over relays: First, why do ...
My technical analysis of NIP-95 and file distribution over relays:
First, why do people want to distribute files directly on relays?
Well, the motivation is pretty straightforward: we want to be able to treat files like notes, meaning:
1. We can verify their authenticity
2. We can distribute them to multiple locations, and re-distribute them if the locations change
3. We can link to them without having to link to a particular hosting provider (that may go down or stop serving the file in future)
In all honestly, I agree that relays (or relay-like systems) are a perfect fit for this type of need, and I think it’s inevitable for relays to eventually start offering some form of file hosting service. Not all, but certainly paid relays might.
NIP-95 is one proposed way to get relays to host files. Dead simple: get the file contents and stick it in a Nostr event, and you’ve got files on relays! Certainly, it satisfies all three needs I mentioned earlier, so what’s wrong with it?
Well, it doesn’t appreciate how different files and notes actually are. Consider these particular behaviours that apply to files but not to notes:
1. Streaming
Files on the web are streamed in, meaning that you can see a low-resolution image well before you’ve finished downloading the full file, or you can watch the beginning of a video or beginning of a sound file without downloading it in its entirety.
If files are embedded as base64-encoded strings inside note content, a lot of this streaming behaviour that is taken care of by the browser needs to be handled manually by clients. You’ll need a streaming JSON parser and a streaming base64 parser, and if you’re lucky the note fields stream in in the right order that you can start showing the note as it’s streamed in (which is not guaranteed).
2. Seeking
Continuing along the streaming idea, files are also seekable. You can fetch a part of a file and download just that.
Again, Nostr notes don’t have such functionality at all. You get them in their entirety from beginning to end. What’s more, video files include seeking metadata at the beginning that tell a video player at what byte offset in the file they can find what second of video, enabling seeking to work. Even if you could request a specific range of a note, you’d have to play some tricks to find the right byte offset in the base64-encoded note contents that corresponds to the right byte of the file. Again, here, this is all work that comes for free in a browser, and it now is left to clients if they want to offer a good user experience.
(For those who are curious about the technical side of this, on browsers who support the MediaSource extension, you can pass a video element your own buffers of video data. So in theory clients could stream in note contents, decode them, and pass them to a video player. However, the MediaSource extension is crucially not supported on iOS, who exclusively permit Apple’s HLS streaming format.)
3. Request cancellation
Finally, to make all this magical video and file streaming work browsers frequently open requests, get what they need, and then drop the request before it’s completed. It’s the final essential behaviour that make the streaming user experience good.
How do you cancel an incoming websocket message? You have to close the websocket.
You’d have to close the websocket, re-open it, re-sub to all the things you wanted from the relay and hope to dear god that you haven’t accidentally re-subbed to the file.
To Conclude
It might sound to some like my critique here is missing the point entirely of NIP-95 files: they’re not supposed to cover all use cases for files and are generally just small images.
And that’s where I’m just not so sure. I think if NIP-95 is spec’d to only be for small files that won’t really upset relays, and that won’t really need streaming, then it’s already lost a lot of its appeal to me. Because that means it will only ever be able to solve for a subset of the problem, and the problem of actual file distribution remains open. What makes Nostr great right now is that everyone is shipping and testing lots of new wild ideas, and we all have a willingness to try and see what works, and adapt, and evolve. What’s more, there’s a certain anti-tradition mentality where nobody’s shy of questioning the established ways of doing things and that’s something that makes me really love developing for this ecosystem.
That applies just the same for file distribution. Maybe the way it’s been done traditionally isn’t THE way to do it, and novel solutions should be considered. But we risk taking steps back by not being aware of how these kind of problems have been solved traditionally and why. HTTP file streaming is very very capable, and we should try to craft something that can preserve all the great benefits of the old method while adding decentralisation and trustlessness.
And for that, I think we need to bring NIP-95 back to the workshop and give it another go. I suggest:
- file delivery happens outside of web sockets over regular HTTP (easy for clients)
- relays who host file attachments can signal to clients inside the protocol where they can go to download them
- files are hashed with a j-hash (shameless plug https://github.com/bmewj/j-hash-node), so that their contents can be verified progressively without having to download the entire file (i.e. when streamed or seeked)
Ok! That’s all. Thanks for reading! :)
Tagging a bunch of people I admire to engage (if they want) in the discussion:
Rabble (npub1wmr…g240)
@npub1gcxzte5zlkncx26j68ez60fzkvtkm9e0vrwdcvsjakxf9mu9qewqlfnj
fiatjaf (npub180c…h6w6)
cameri (npub1qqq…n47m)
anton (npub1h72…k5rt)
Martti Malmi (npub1g53…drvk)
First, why do people want to distribute files directly on relays?
Well, the motivation is pretty straightforward: we want to be able to treat files like notes, meaning:
1. We can verify their authenticity
2. We can distribute them to multiple locations, and re-distribute them if the locations change
3. We can link to them without having to link to a particular hosting provider (that may go down or stop serving the file in future)
In all honestly, I agree that relays (or relay-like systems) are a perfect fit for this type of need, and I think it’s inevitable for relays to eventually start offering some form of file hosting service. Not all, but certainly paid relays might.
NIP-95 is one proposed way to get relays to host files. Dead simple: get the file contents and stick it in a Nostr event, and you’ve got files on relays! Certainly, it satisfies all three needs I mentioned earlier, so what’s wrong with it?
Well, it doesn’t appreciate how different files and notes actually are. Consider these particular behaviours that apply to files but not to notes:
1. Streaming
Files on the web are streamed in, meaning that you can see a low-resolution image well before you’ve finished downloading the full file, or you can watch the beginning of a video or beginning of a sound file without downloading it in its entirety.
If files are embedded as base64-encoded strings inside note content, a lot of this streaming behaviour that is taken care of by the browser needs to be handled manually by clients. You’ll need a streaming JSON parser and a streaming base64 parser, and if you’re lucky the note fields stream in in the right order that you can start showing the note as it’s streamed in (which is not guaranteed).
2. Seeking
Continuing along the streaming idea, files are also seekable. You can fetch a part of a file and download just that.
Again, Nostr notes don’t have such functionality at all. You get them in their entirety from beginning to end. What’s more, video files include seeking metadata at the beginning that tell a video player at what byte offset in the file they can find what second of video, enabling seeking to work. Even if you could request a specific range of a note, you’d have to play some tricks to find the right byte offset in the base64-encoded note contents that corresponds to the right byte of the file. Again, here, this is all work that comes for free in a browser, and it now is left to clients if they want to offer a good user experience.
(For those who are curious about the technical side of this, on browsers who support the MediaSource extension, you can pass a video element your own buffers of video data. So in theory clients could stream in note contents, decode them, and pass them to a video player. However, the MediaSource extension is crucially not supported on iOS, who exclusively permit Apple’s HLS streaming format.)
3. Request cancellation
Finally, to make all this magical video and file streaming work browsers frequently open requests, get what they need, and then drop the request before it’s completed. It’s the final essential behaviour that make the streaming user experience good.
How do you cancel an incoming websocket message? You have to close the websocket.
You’d have to close the websocket, re-open it, re-sub to all the things you wanted from the relay and hope to dear god that you haven’t accidentally re-subbed to the file.
To Conclude
It might sound to some like my critique here is missing the point entirely of NIP-95 files: they’re not supposed to cover all use cases for files and are generally just small images.
And that’s where I’m just not so sure. I think if NIP-95 is spec’d to only be for small files that won’t really upset relays, and that won’t really need streaming, then it’s already lost a lot of its appeal to me. Because that means it will only ever be able to solve for a subset of the problem, and the problem of actual file distribution remains open. What makes Nostr great right now is that everyone is shipping and testing lots of new wild ideas, and we all have a willingness to try and see what works, and adapt, and evolve. What’s more, there’s a certain anti-tradition mentality where nobody’s shy of questioning the established ways of doing things and that’s something that makes me really love developing for this ecosystem.
That applies just the same for file distribution. Maybe the way it’s been done traditionally isn’t THE way to do it, and novel solutions should be considered. But we risk taking steps back by not being aware of how these kind of problems have been solved traditionally and why. HTTP file streaming is very very capable, and we should try to craft something that can preserve all the great benefits of the old method while adding decentralisation and trustlessness.
And for that, I think we need to bring NIP-95 back to the workshop and give it another go. I suggest:
- file delivery happens outside of web sockets over regular HTTP (easy for clients)
- relays who host file attachments can signal to clients inside the protocol where they can go to download them
- files are hashed with a j-hash (shameless plug https://github.com/bmewj/j-hash-node), so that their contents can be verified progressively without having to download the entire file (i.e. when streamed or seeked)
Ok! That’s all. Thanks for reading! :)
Tagging a bunch of people I admire to engage (if they want) in the discussion:
Rabble (npub1wmr…g240)
@npub1gcxzte5zlkncx26j68ez60fzkvtkm9e0vrwdcvsjakxf9mu9qewqlfnj
fiatjaf (npub180c…h6w6)
cameri (npub1qqq…n47m)
anton (npub1h72…k5rt)
Martti Malmi (npub1g53…drvk)