Pool DNS zones stuck

I get the impression that the pool DNS zones are currently “stuck”. Servers with a score less than 10 may still be included in the pool DNS if their score has dropped only recently below 10, and I’d imagine the reverse is also true, ie. newly added servers won’t be included in the pool DNS even if their score increases to over 10.

Evidence: My server 95.111.202.5 is included in the cn DNS even though the last time its score was above 10 was at 2026-04-22 07:10:23 UTC (around 8 hours ago as of now).

$ dig 0.cn.pool.ntp.org

0.cn.pool.ntp.org. 45 IN A 95.111.202.5

$ TZ=UTC date
Wed Apr 22 15:14:36 UTC 2026

Monitoring seems to work, but changes to server’s score status (transition to over 10 or transition to less than 10) do not seem to have an effect.

I guess I’ll need to ping @ask about this.

I’ve been watching the PH zone for around an hour now but this new server hasn’t shown up yet.

@marco.davids can soon report if his new server starts getting the usual amount of pool traffic or not. As of now, its score is 9.2 and increasing.

I also set some of my other servers to monitoring mode only, but I don’t see a matching decrease in packet rates.

I also noticed some strange changes in traffic patterns these recent days, which might be due to the same reason: servers not getting moved in and out of zones as they should be.

E.g., a server in the HK and TW zones that always sees most traffic geolocated to China, but at a reasonable level (> 10Mbit/s), and manageable via the netspeed setting, suddenly saw 5 minute average peaks above 100Mbit/s even at the smallest allowed non-zero netspeed setting in the last few days. Apart from that overloading the single core machine, I had to drastically throttle incoming traffic not use up the remainder of my monthly traffic quota in a few days (and temporarily put the server in monitoring-only whenever things were getting out of hand). Now, it seems back to normal, though, i.e., I will slowly increase the netspeed again from the current setting of 512kbit.

I can confirm that at score 10.2, the newly added servers are not yet receiving any NTP queries.

I think increasing the netspeed won’t increase the traffic in this situation, but I’d be happy to be proven wrong.

It was set at the maximum value of 3 Gbit when I added the server.

I set the netspeed to a value that usually would kill the server a few minutes ago, and indeed no change whatsoever so far, thus unfortunately not proving you wrong :wink:. But there also is a strong diurnal pattern in those zones, with this currently being the quieter part of the day over there. So I’ll set it back down now, and then try again in the morning our time so I can control the traffic in case things start getting out of hand again (though I would still have expected some change at least… :thinking:).

Ask is currently still in the process of migrating functionality from Equinix Metal to new infrastructure, hoping to be done with the cluster migrations this week or next. I guess at least some of the various infrastructure-related issues seen these days might be due to those activities.

Just checked: The DNS zone data apparently was last updated 7 hours, 4 minutes ago (as of me writing this). That would explain a few things…

Where can I find the ‘DNS status’ ?

Sorry, had meant to include a link in my previous post, but forgot:

Just sent an e-mail to @ask just in case he is not aware yet…

On the upside: I can set the netspeed on an IPv4 server in Asia to 3Gbit without it going up in smoke within minutes… :joy:

Thanks!

Interesting, that page has a link at the bottom that leads to nowhere.

There is a PR to have that removed. But I’ll also try to see whether I can figure out where this was intended to go, maybe that function can be restored, and the link re-added accordingly.

status.ntppool.org shows a “disconnect” in DNS:

Still ongoing…

The DNS data was updated 1 days, 10 hours, 39 minutes ago (current time: 2026-04-23T13:24:01)

I think it is fair to say that by now this is becoming a pretty serious issue.

The pool is being used by hundreds of millions of systems around the world. It’s the default “time server” for most of the major Linux distributions and many networked appliances

Glad I am not one of them. GPS+PPS, NIST, USNO

The example server of mine that I linked to in the first message sends normally around 165 GB of NTP traffic each day. Yesterday that total figure was 404 GB. At that rate I’ll burn through my monthly traffic quota very quickly. Normally that server’s netspeed setting is 100 Mbit/s but I decreased it to 512 Kbit/s when I ran into this issue. Sadly that setting has no effect at this moment.

The traffic continued at that rate up to around an hour ago, when my ISP apparently thought my server was under DDoS and null routed its IPv4 traffic. I can kind of understand their thinking. Normally null routing the traffic would also make the pool monitors drop the server from the pool DNS and the traffic would slow down, but as the pool DNS zones are stuck, the ISP will continue receiving tons of traffic (like 6 MB/sec) until this issue gets fixed. The server won’t see this incoming traffic until the ISP drops the null routing, and I won’t be asking the ISP about this until the pool DNS issue is fixed.

Those who have IPv6 can still view that server’s statistics. Apparently this server can handle 70k queries per second at around 85% CPU utilization.

Then there’s the issue of servers not getting promoted/demoted to/from pool DNS in case of new servers, or the server’s time being wildly off, for example. I’m hoping that this issue gets fixed soon.

Looking better now! :star_struck:

The DNS data was updated 0 hours, 2 minutes ago (current time: 2026-04-23T18:32:07).