Certain servers are not replying

Aren’t most people behind a CGNAT these days? We used to have a public IP address 10 years ago. Now it has a premium price.

I don’t know any better than you do in this regard. I appreciate you helping out on this issue though.

Doesn’t seem to be the case where I live, in Norway. But maybe it’s more common in Asia than Europe?

I looked at a 60-second pcap from Sebhoster’s server. Typical caveats: only examined one server for a short time, traffic may vary over time, etc.

Preliminary analysis below. The behavior is unlike other high-request rate systems that I’ve looked at.

NTP requests were sent by 17,564 clients during the 60-seconds. 99.6% of the requests were sent by 152 clients! Most clients sent requests at a steady rate of 300-1500 during the entire period.

This is probably not a NAT/VM related situation.

Except for the absolute rate, all 152 clients exhibit similar behavior: UDP source port is constant, the requests show alarm=0, version=4, stratum=0. The NTP request transmit time (set by the client when it sends the request) are all in error by either 20.5, 41, or 82 years.

Looking again at the NTP request transmit time stamp the fractional part is always <= 0.232 seconds. A similar behavior was seen in recent years due to a bug in systemd’s timesyncd service. The Philippines situation differs in other aspects.

My guess is buggy client software is to blame.

2 Likes

Just block these clients?

That only solves the symptom, not the problem. And it would need to be done by every operator of any pool-joined server in the Phillipines.

I think I’ll start writing emails to the providers responsible for the IP addresses and see if that helps

Hi o/

Did you have any reply to your emails?

We’re still having the same problem:

# ntpdate -d 0.rhel.pool.ntp.org
mail.fortunetobacco.com forward check lookup fail: Success
host found : 222.127.1.19 (mail.fortunetobacco.com)
transmit(222.127.1.19)
transmit(222.127.1.20)
transmit(222.127.1.27)
transmit(222.127.1.18)
transmit(222.127.1.19)
transmit(222.127.1.20)
transmit(222.127.1.27)
transmit(222.127.1.18)
transmit(222.127.1.19)
transmit(222.127.1.20)
transmit(222.127.1.27)
transmit(222.127.1.18)
transmit(222.127.1.19)
transmit(222.127.1.20)
transmit(222.127.1.27)
transmit(222.127.1.18)
222.127.1.19: Server dropped: no data
222.127.1.20: Server dropped: no data
222.127.1.27: Server dropped: no data
222.127.1.18: Server dropped: no data

Can these servers be removed from the pool?

Thanks in advance.

I ran mtr (traceroute) from New York City for two of these server addresses: 122.127.1.20 and 122.127.1.22. Both addresses showed the same behavior:
Target port 123 (NTP) No packets reach server
Target port 122 or 124 At least 80% of the packets reach NTP server
Ports 122 and 124 are used for comparison.
It looks like AS4775 is intentionally blocking NTP messages from New York.

I checked NTP connectivity from another client located in suburban Chicago and saw zero loss (NTP connectivity was 100%)

Last night from ~200 vantage points in the Philippines and beyond towards 122.127.1.26: not a single response.

At the same time, the score was 20. But it is interesting to see that multiple monitors have score -100.

Looks indeed like there is heavy selective filtering.

New measurement with the same vantage points, and some more, probing 122.127.1.26 once an hour for the next 24 hours to see whether there is any variation throughout the day.

Works fine here:

root@workstation:~# ntpdate -q 222.127.1.26
server 222.127.1.26, stratum 2, offset +0.004966, delay 0.34708
21 May 18:06:19 ntpdate[20110]: adjust time server 222.127.1.26 offset +0.004966 sec

I know the system worked in the past, but seeing no response whatsoever got me wondering. But it does work.

Maybe networks are blocked because of bigh number of requests?

My server does this if an IP(gateway) is polling too much.

It simply stops responding.

I see ISP’s request massive over 1 IP to my server, they are out of luck and block themselves.