Nuh 🔻 on Nostr: I didn't say anything about crucial, I personally like small rooms and discord ...
I didn't say anything about crucial, I personally like small rooms and discord servers that is fine. But definitely the claim that you can do with Nostr what you can do with centralised search indexes Without the centralisation is a prevalent and unchallenged.
Mainline approach only works if your query is a hash, and only a hash. And the data is small enough and the nodes are numerous enough that you can keep publishing the data cheaply with redundancy factor.
For general purpose search, the query can't be just a hash (if it is just use Mainline I guess) and the data is usually too large for it to be cheap to store and serve (which means less nodes).
So you can't shard the data, and even if you can there is too much of it.
Pkarr idea is to use Mainline for the absolute minimum, and pray that is not too much, and cache at every possible layer to reduce the load on the public common, otherwise we have unrealistic expectations
Mainline approach only works if your query is a hash, and only a hash. And the data is small enough and the nodes are numerous enough that you can keep publishing the data cheaply with redundancy factor.
For general purpose search, the query can't be just a hash (if it is just use Mainline I guess) and the data is usually too large for it to be cheap to store and serve (which means less nodes).
So you can't shard the data, and even if you can there is too much of it.
Pkarr idea is to use Mainline for the absolute minimum, and pray that is not too much, and cache at every possible layer to reduce the load on the public common, otherwise we have unrealistic expectations