mleku on Nostr: right, but for how long?, is the client gonna be nice and drop it when the app is not ...
right, but for how long?, is the client gonna be nice and drop it when the app is not in focus or mouse over after a little while? can't trust it, it's a real problem
what i'm thinking is that there needs to be a garbage collector so the open states needs a background watcher that watches the watchers and pulls the plug on the oldest ones after too many stay open from one IP and/or in total
but the database thread does not need to be multithreaded, and anyway, the database i'm using, badger, it has a batch concurrency API built into it, i used it to do the GC census, takes 8 seconds on a 15gb event store, which i think is pretty damn fast, but again probably gonna have problems with that at 150gb, so probably need to have a periodic full rescan and a table dedicated to tracking this access data more efficiently
scaling is hard, really, as is managing competing use of resources
what i'm thinking is that there needs to be a garbage collector so the open states needs a background watcher that watches the watchers and pulls the plug on the oldest ones after too many stay open from one IP and/or in total
but the database thread does not need to be multithreaded, and anyway, the database i'm using, badger, it has a batch concurrency API built into it, i used it to do the GC census, takes 8 seconds on a 15gb event store, which i think is pretty damn fast, but again probably gonna have problems with that at 150gb, so probably need to have a periodic full rescan and a table dedicated to tracking this access data more efficiently
scaling is hard, really, as is managing competing use of resources