mleku on Nostr: it uses a lot of CPU because it's doing a massive amount of hashing of content to ask ...
it uses a lot of CPU because it's doing a massive amount of hashing of content to ask other strfry relays if they have "this hash" and "that hash".
this is the inefficiency that HORNET Storage is aiming to eliminate.
i dunno if posting content to other relays that "didn't ask for them" is entirely a good strategy but it's because of this promiscuity that you can find all the posts in a thread where users are outside your relay set, but in the set of those in the thread.
it would be better if relays had some system to quickly identify missing content that users are requesting and cache enough of it ahead of time that the user is none the wiser. Negentropy is one effort in this direction, but it is very primitive. HORNET Storage FOREST algorithm reduces the computation and network traffic requirement by the log of the number of accounts involved.
without this nostr would rapidly silo up on the most popular relays who would then either fail due to insufficient capacity or insifficient resources to pay for capacity, and only the big ones would stay operational and provide network data. and then the only thing we have is yet another facebook, yet another twitter. before you know it there will be ads forced on you and KYC, the whole nine yards. it's naive to think that the big guys aren't seeing this as a way to rebrand themselves as "open" with very little resources spent to take it over, and no real difference, since they can apply their algorithms to squash all the "offensive" content, shadowban those users by not relaying their posts, and colluding with other big relays to maintain a unified front, and keep their revenue stream from government and corporations.
this is the problem that we are warning people about that nobody seems to want to acknowledge who are supposedly the "wizards of nostr". anyone with a fragment of knowledge about distributed systems database state replication can tell you this is a looming problem.
it would be better if it was driven by an efficient algorithm that identifies relevant users content and could efficiently acquire the data after it has been requested, to proactively get more of that content from nearby it in the graph.
this will struggle to happen while everyone's watching the big swinging dicks at the top of the trending graph and paying no attention to the gradual attrition of small relays on the margin as the data set gets beyond the resources of volunteers.
i dunno how to explain it any more concisely, but due to what i see going on in the protocol and the way the social control is trumping the market needs, someone needs to start a call for more work to be done on building a scalable consensus.
personal relays really are just caches, they are an important measure in the efforts but also the easiest part to solve. cache syncing, data reference schemes, and ancilliary content hosting are all bigger, much harder problems to solve that are rapidly converging on cloud solutions and you know where that leads.
this is the inefficiency that HORNET Storage is aiming to eliminate.
i dunno if posting content to other relays that "didn't ask for them" is entirely a good strategy but it's because of this promiscuity that you can find all the posts in a thread where users are outside your relay set, but in the set of those in the thread.
it would be better if relays had some system to quickly identify missing content that users are requesting and cache enough of it ahead of time that the user is none the wiser. Negentropy is one effort in this direction, but it is very primitive. HORNET Storage FOREST algorithm reduces the computation and network traffic requirement by the log of the number of accounts involved.
without this nostr would rapidly silo up on the most popular relays who would then either fail due to insufficient capacity or insifficient resources to pay for capacity, and only the big ones would stay operational and provide network data. and then the only thing we have is yet another facebook, yet another twitter. before you know it there will be ads forced on you and KYC, the whole nine yards. it's naive to think that the big guys aren't seeing this as a way to rebrand themselves as "open" with very little resources spent to take it over, and no real difference, since they can apply their algorithms to squash all the "offensive" content, shadowban those users by not relaying their posts, and colluding with other big relays to maintain a unified front, and keep their revenue stream from government and corporations.
this is the problem that we are warning people about that nobody seems to want to acknowledge who are supposedly the "wizards of nostr". anyone with a fragment of knowledge about distributed systems database state replication can tell you this is a looming problem.
it would be better if it was driven by an efficient algorithm that identifies relevant users content and could efficiently acquire the data after it has been requested, to proactively get more of that content from nearby it in the graph.
this will struggle to happen while everyone's watching the big swinging dicks at the top of the trending graph and paying no attention to the gradual attrition of small relays on the margin as the data set gets beyond the resources of volunteers.
i dunno how to explain it any more concisely, but due to what i see going on in the protocol and the way the social control is trumping the market needs, someone needs to start a call for more work to be done on building a scalable consensus.
personal relays really are just caches, they are an important measure in the efforts but also the easiest part to solve. cache syncing, data reference schemes, and ancilliary content hosting are all bigger, much harder problems to solve that are rapidly converging on cloud solutions and you know where that leads.