I just got email that my server is scheduled for deletion from the beta NTP pool system. On beta, I have about -50 score form both monitoring locations, but from real site, I have score >15. I am the only one with this issue?
Why the same monitoring location is able to connect to my server in real pool, but not in Beta?
The logs says you are rate limiting – the beta system is doing a few more queries (all should be 2 seconds apart) when testing.
There may be a network problem near your server; I’ve asked for a packet capture.
One NTP request results in multiple responses. This occurs for both the IPv4 and IPv6 server addresses. I’ve seen this from multiple client sites, so the duplicates are not related to the NTP pool monitors.
Furthermore I believe that the long-observed network packet drops are occurring.
@ask I’m not doing any special rate limiting, so it must be default ntpd. Should I somehow change the “discard” config option? I have updated debian on the server about 2 weeks ago, now ntpd is version 1:4.2.8p10+dfsg-3+deb9u2.
I’m not sure about multiple responses to one request. For example the last one from the dump, I see 2 requests from port 35085 and 2 answers from my server and the 2nd answer got port unreachable from the remote machine…
22:00:05.509595 IP 184.108.40.206.35085 > 220.127.116.11.123: NTPv4, Client, length 48 22:00:05.509674 IP 18.104.22.168.123 > 22.214.171.124.35085: NTPv4, Server, length 48 22:00:05.511739 IP 126.96.36.199.35085 > 188.8.131.52.123: NTPv4, Client, length 48 22:00:05.511797 IP 184.108.40.206.123 > 220.127.116.11.35085: NTPv4, Server, length 48 22:00:05.612015 IP 18.104.22.168 > 22.214.171.124: ICMP 126.96.36.199 udp port 35085 unreachable, length 84
Here’s what happened, based on the the two tcpdumps. I’ll list the Newark side below, times are UTC
NTP monitor sent one request
2020-05-20 20:00:05.453937 188.8.131.52 -> 184.108.40.206 NTP 90 NTP Version 4, client
That request was duplicated in the IP network between the monitor and your NTP server (more below).
There are now two requests en route to your server. Both requests have the same NTP “Transmit time stamp.”
First response arrives and NTP server responds:
2020-05-20 20:00:05.555389 220.127.116.11 -> 18.104.22.168 NTP 90 NTP Version 4, server
NTP pool monitor software closes UDP port 35085
Second request arrives and NTP server responds
2020-05-20 20:00:05.557512 22.214.171.124 -> 126.96.36.199 NTP 90 NTP Version 4, server
Since the NTP Pool monitor has closed the port the incoming packet is rejected:
2020-05-20 20:00:05.557535 188.8.131.52 -> 184.108.40.206 ICMP 118 Destination unreachable (Port unreachable)
It might be easier to see this by comparing tcpdumps; that’s why I requested a sample pcap from your side.
I have some extra NTP probes running now, so this would be a great time to collect the tcpdump.
Tentatively I’ve identified the duplicates as originating from “Lightstorm”; I’m unfamiliar with this ISP.
Ok, I have started another tcpdump now.
Lightstorm is the serverhousing ISP where this server is physically located. We have entire subnet 220.127.116.11/25 from them.
Sorry, I have found old rule in iptables dropping NTP packets from IP if more than 4 arrive in 4 seconds. I have now changed it to 8/8. Let’s see if it will help.