mleku on Nostr: i was just watching #realy get hammered by yet another spider as i was tinkering with ...
i was just watching #realy get hammered by yet another spider as i was tinkering with ... idk, something... and i wanted to check up on how this rate limiter was supposed to be set up
turns out i had it in there, but was not triggering the wait timer it has for unauthed requests
it is now triggering, and spiders get very slow responses... 0.033 responses per second, or in other words one every 30 seconds with a burst of 5 (meaning it can do 5 in thirty seconds but then not again until 30 seconds passes
the logs are soso much quieter now haha
it's public readable, but the "public" does not include spiders
if the spider tolerates this lag, then ok, they can have their 6 requests per minute
my spider drops a connection if it notices that the within the last 5 requests it is being limited
that's called "manners" but that's why we have rate limiters and that's why i'm pleased to announce that #realy actually has one now
i should add a pair of options to configure the rate and burst limits... what i have set is pretty aggressive really, but fuck the spiders
many spiders on nostr are just gathering social graph traffic data, if you ask me, figuring out who is talking to who because most relays give out DM events to anyone
turns out i had it in there, but was not triggering the wait timer it has for unauthed requests
it is now triggering, and spiders get very slow responses... 0.033 responses per second, or in other words one every 30 seconds with a burst of 5 (meaning it can do 5 in thirty seconds but then not again until 30 seconds passes
the logs are soso much quieter now haha
it's public readable, but the "public" does not include spiders
if the spider tolerates this lag, then ok, they can have their 6 requests per minute
my spider drops a connection if it notices that the within the last 5 requests it is being limited
that's called "manners" but that's why we have rate limiters and that's why i'm pleased to announce that #realy actually has one now
i should add a pair of options to configure the rate and burst limits... what i have set is pretty aggressive really, but fuck the spiders
many spiders on nostr are just gathering social graph traffic data, if you ask me, figuring out who is talking to who because most relays give out DM events to anyone