Hi, i’m experiencing a lot of IA scrapers in my server that is affecting QoS and because of that, i enabled blocking TopSpammers in pfBlocker-NG which is the pf-firewall in pfSense. After i did that, NTP server monitors started reporting my NTP unreachable. Is there any whitelist y can add to my firewall or at least the country of origin so i can disable TopSpammers for that country? Or does the system uses other monitors which are not affected?
Cheers.
Don’t block all the traffic from the scraping bots, but only connections to your server’s TCP ports 80 and 443 from those IP addresses.
That said, it is unlikely that you would block any monitors by blocking web scraping IP addresses. There’s something else wrong in your firewall setup.
Whitelisting monitor IP addresses is a really bad idea.
I cannot block on ports using quick tables, if i can just get a monitor IP that reports the fail i can pinpoint what blocklist is causing the block and simply disable it. Is there anyway to get info on only 1 ip address failing? you can send me on private message.
Cheers
If you want to lift the current restriction for some specific monitor’s country, do note that the country name is included in the monitor’s name. For example, fihel3-2trgvm8 is in Finland (fi) near the Helsinki (hel) airport.
I’m not familiar with the mentioned TopSpammers, but one alternative might be to not use that TopSpammers thing at all. Instead, I’d set up some other firewall rule to limit the number of new incoming connections to TCP ports 80 and 443 to some reasonable value.
Are you using NAT? As I have/had the same problems.
NAT tables get overloaded and your other services go down.
In my case it’s NTP-behind NAT where NAT keeps tracking the state of the port, where it should be stateless.
Not many routers can handle this.
The issue has been fixed, the CPU of the pfSense firewall has been upgraded to one with the double muscle power and everything is going very smooth. I have A rating for buffer bloat and the server reached again 20.0 score. Now, i think there is a server in NL? that is giving very bad score.
| nlein1-2ze617b | -85.3 |
|---|
I disabled all the additional blocklists and only the basic priority 1 is enabled, which cannot be disabled for security reasons. We have no blocks to that country either. Maybe it will fix itself someday?
Cheers.