You can have a relay hint for each tag in kind:3 follow lists, some clients have it there, there is no reason why you can't have the same for the tags in the follow pack.You can't have this local database thing in all contexts like all microapps and whatnot (those can probably just use a naïve NIP-65 approach) but a full-fledged Nostr client should.
But you can't just add all these relays indiscriminately into a flat list and start querying them all -- or maybe you can, but that's very inefficient. It's better to have always a handy list of 2 or 3 relays that are "optimal" for each pubkey and only query their posts from those.
The best way I came up with (well, more reasonably I stole the idea from Mike Dilger ☑️ (nprofile…442w)) is to have a local database (see https://github.com/nbd-wtf/go-nostr/blob/master/sdk/hints/interface.go) that associates relays with pubkeys that tracks the last timestamp we saw a hint from for that specific pubkey on that specific relay. Hints can come from other people's event tags, nprofile and nevent codes seen in the wild, nip05 and nip65 relay lists, then there is also a timestamp for the latest successful event fetched from and the last attempt.
There are different weights for each of these hints such that when we add them all up the nip65 list will get a higher score, but failed attempts will get heavily discounted so if a person stops publishing on a relay but we keep trying to get their posts from there eventually that relay will lose score.
The nprofile or event tag hints have very little weight but they serve at least to point to relays we can target briefly if the others stop working so clients can recover even from the most brutal forms of censorship.
I tried to illustrate that process at https://how-nostr-works.pages.dev/#/pathological
fiatjaf on Nostr: This may interest some people, it's a brief explanation of how I think outbox should ...
This may interest some people, it's a brief explanation of how I think outbox should work ideally (criticism and suggestions welcome):