kjeldahl on Nostr: When writing my first Nostr apps I had recurring issues with notes disappearing and ...

When writing my first Nostr apps I had recurring issues with notes disappearing and user metadata and profiles not being up to date. It took a while, but eventually I figured out how relays work, and how it's tied together with user profiles and other events containing relevant data (preferred relays etc). I'm still not sure what the "ideal" way to retrieve initial user data etc is. Which relays do I query? Do I actually have to implement a "Nostr crawler" that continuously monitor events to detect relays, and asking relays more or less at random for data that I have been unable to retrieve? Besides the relay issue, which can be solved by some more or less hardcoded list of popular relays, most things are working great now. I typically cache events when found. Events that can be "renewed" I periodically try to refresh (pubkey+kind), but client never have to wait. When clients retrieve something that can theoretically be updated, I return the last known value. But if some value haven't been tried to be re-fetched in a while, I put it in a queue to be re-acquired by another "worker" process. So clients will never have to wait for anything to be fetched from relays, unless it's something that has never been seen before of course. There's no manual for how this works AFAIK, so there are many pieces to work out by yourself. You need some ability and lots of curiosity IMHO.