Stefano Marinelli on Nostr: This morning, an e-commerce site (built on Laravel and well-developed, hence quite ...
This morning, an e-commerce site (built on Laravel and well-developed, hence quite efficient) started showing signs of slowing down. This had also happened a few weeks ago, and we partially managed the situation by increasing the VPS power and freeing up the physical machine from other loads.
An analysis of the nginx log reveals that the server is being bombarded with requests from Bytedance. As often happens in these cases, I attempted to firewall the IPs associated with the bots.
However, as soon as I block one IP, (or entire class) the crawling resumes (violently, almost like triggering a DoS attack) from another IP on another class.
They don't respect the robots.txt file.
The IPs they use online don't match the ones from which the requests originate; they probably constantly acquire and change IP blocks.
It makes me wonder: if everyone online behaved like this, everything would collapse in a matter of minutes.
#webdevelopment #serverissues #DDoS #Bytedance #firewall
An analysis of the nginx log reveals that the server is being bombarded with requests from Bytedance. As often happens in these cases, I attempted to firewall the IPs associated with the bots.
However, as soon as I block one IP, (or entire class) the crawling resumes (violently, almost like triggering a DoS attack) from another IP on another class.
They don't respect the robots.txt file.
The IPs they use online don't match the ones from which the requests originate; they probably constantly acquire and change IP blocks.
It makes me wonder: if everyone online behaved like this, everything would collapse in a matter of minutes.
#webdevelopment #serverissues #DDoS #Bytedance #firewall