NTP traffic drop for .de server

I have a NTP server in the .de zone. From 29.05-30.05 and till then the NTP IPv4 traffic dropped significantly. Is there a reason for this and is it expected? The monitoring rating is always 20.

Let me second that, I encountered the same thing:

Strange. Mine looks consistent. Located in the US region

Nothing changed in Taiwan.
Legend in Chinese but same as above↑ English chart
(Legend in Chinese but same as above↑ English chart by @NTPman )

1 Like

This may be slightly off-topic but below is the normal traffic (>50k requests / second) between 7-11pm China time for HK and ASIA zones for me.

image

That’s pretty wild. No obvious changes in the available servers for Germany or Europe around that time.

My server is in Switzerland. But at least Europe. The other change, that the traffic is not so spiky, it is more flat now.

Ooh, …

Over a couple days that week (around May 29-30) I did disable two DNS servers that were running a very old version of the GeoDNS software (all the other servers are managed by @gfk and I so we keep them updated).

I have to look through the change history to see if I can think of why that’d make such a change though.

Another drop today. The previous one was also around 2 am (midnight in UTC).

Hm, I saw the first drop on my server in the de zone, but nothing today.

The query rate to the DNS servers (globally and from Switzerland) haven’t had any changes. I don’t log the answers the DNS servers give, but that might be a worthwhile addition to help validate that future changes work as expected (and it could maybe help figure out things like this).

I monitor a New Zealand IPv4 NTP server’s traffic closely and noticed the same decrease in request rate at the end of May. I compared two 24-hour intervals, one before and one after, the change.
Summary:

  • the number of unique client IPs dropped by 40%
    The decrease came from non-New Zealand client IPs.
  • The number of non-New Zealand clients dropped significantly.
    The number of requests from New Zealand clients increased a bit.
  • No observed change in per-client behavior

This seems consistent with a GeoDNS change, perhaps one specific to the global zone?

Did anyone with NTP servers in under-served areas noticed a traffic increase?

1 Like

FWIW, trying to divine information from a US server’s network graphs, IPv6 traffic patterns changed a bit at the same time. It looks like peak times (around 01:00 UTC) increased by around 20-30 Kbps and the troughs (around 10:00) might have stayed about the same or increased slightly.

IPv4 might also have increased a bit but it’s harder to say.

Also a drop in Moldova & Lithuania

Moldova :

Lithuania:

My servers in Mumbai get a bit more traffic since end of may. This graph is showing accumulated queries over 3 servers.

Since my servers in DE got quite notable amounts of traffic from Asia (mostly China), a change in GeoDNS might be a good explaination. It makes sense for GeoDNS to route Clients from Asia to Servers on the same continent.

Some more data supporting the GeoDNS hypothesis. I have data from one day in early May and one day in early June from NTP servers located in San Francisco, Washington, and London.
NTP requests from
Chinanet(AS4134) dropped by 30%
Argentina(AS27747) dropped by 99%
Emirates (AS5384) dropped by 50-70%)

1 Like

The second drop wasn’t that significant, the traffic is slowly climbing up:
image

I also assume that this change was for the better for the pool.

In this thread Far away clients from AWS I mentioned in 2022 that I see a lot of traffic from AWS virtual machines from far away. This is now not the case any more after capturing the requests for a few minutes.

So I assume the GeoDNS setup before the change was working suboptimal for some clients.

1 Like

I was debugging my box for like forever as I suspected a hardware or network config or debian upgrade or ISP issue since my connections dropped too … guess I was just wasting my time.

image

(server is pool.ntp.org: Statistics for 185.144.161.170)