Hello. Im back, I was very active about 5 years ago trying to get enough servers for the CN pool. An for a few years it was working. I still have 8 ntp servers in pool and alot better hardware than before. But during peak hours, that isnt enough. Most likely, the demand is also higher in 2024. And the scoring system doesnt take into count that the CN zone is underserved and when my servers gets scores below 10 and are kicked out. And the rest will take the hit. And get kicked out. And after some hours its just a few servers or 0 servers in the cn.pool.ntp.org pool. And if my server gets obove score 10 again I will be the only ntp server for ENTIRE china. About 500-1000 Mbit/s of NTP traffic! Not very funny and this is starting to become I real problem as I cant work for a while. Its not acceptable and I cant have it like this. I have lower the bandwith on my servers and some days it works fine, but its just random when it does and when it doesnt. My suggestions of howto fix this 1 My servers are in zones “@ cn europe se” mostly. If I could differentiate the bandwidth on different zones I could for example have @ - 1 Gbit europe - 1 Gbit se - 1 Gbit cn - 10 Mbit I have much capacity that arent used that can be used in other zones instead. 2 It seems like peak times are 10:00 → 16:00 CET time. Around 14:00 CET are the worse time. I have just accepted that the zone might not work during the entire day, the resources needed for that just isnt here. But I dont want to have my connection “DDoS”:ed because of it. So I want to opt-out from the CN zone with my servers during that time. Can something like this fixed? 3 More forgiving scoring system for the CN zone so everyone arent kicked out. Its counterproductive. 4 If nothing of the above or any other suggestion that achieve that my connection isnt killed because of the mount of ntp packets I must ask to have my servers removed from the CN zone completely. Very sorry but I cant have it like this. Or ask, you can give me back the admin rights so I can remove them myself. And maybe add to some other pool that I can help but doesnt have so much traffic as the CN pool. @ask - or anyone else have any other suggestion of howto solve this?

Sounds good. Pressing both thumbs that it will work in the future. Long storry - short: I’ve asked a big company which is already present in the pool for some help and they saied: yes, sure :slight_smile:

[image] PoolMUC: I think it’s easy, just add a batch of CNAME RR from all countries or region subdomains to the global zone. According to the DNS standard, all clients will use global zone when they try to use region zone. This sounds like a good idea actually!

[image] Coelacanthus: just add a batch of CNAME RR from all countries or region subdomains to the global zone If it were a static zone, that would be true. But the zones are dynamically generated by the pool infrastructure as servers come and go, or are reconfigured. So that code would need to be adapted to take that into account. Also, some other technical and non-technical questions need to be considered as well. E.g., as mentioned, other zones have smaller servers operating without issue because of a “good” client to server ratio in those zones. It needs to be managed carefully not to overload them, as could likely happen if the load of some larger zones is suddenly hitting the wider population with the flip of a switch. A more gradual approach is needed, e.g., by more and more people over time switching to the global zone. Or the infrastructure having a mechanism to slowly ramp up the share of traffic distributed outside the originating zone over time.

[image] PoolMUC: Also, some other technical and non-technical questions need to be considered first. E.g., as mentioned, other zones have smaller servers operating without issue because of a “good” client to server ratio in those zones. It needs to be managed carefully not to overload them, as could likely happen if the load of some larger zones is suddenly hitting the wider population with the flip of a switch. A more gradual approach is needed, e.g., by more and more people over time switching to the global zone. Or the infrastructure having a mechanism to slowly ramp up the share of traffic distributed outside the originating zone over time. For this, I think it’s a good beginning to allow server owner modify the zones of their servers in Management Panel.

[image] Coelacanthus: I think it’s a good beginning to allow server owner modify the zones of their servers in Management Panel Sure, I’d welcome such a change, too. But given the bottleneck in getting things implemented, if it is a question of what to get done first if not everything can be done at the same time, I’d rather have what looks more like a thorough fix implemented first, rather than what seems more of a temporary patch (though that is still relevant later even if the “fix” is done first). At this point, I’d concur to be happy if anything could be done, and if the “fix” is “too big” to get done anytime in the “near” future (and “too big” referring to any number of issues, be it conceptual topics being too complex to be resolved soon, be it the amount of code changes that would be needed in one go, or anything else), I’d be happy to see the “temporary patch” done instead first. I think I have an appreciation on a qualitative level of the potential challenges that come with any change in the system, but I am far away from claiming any sufficient understanding for a quantitative assessment of efforts and impact on prioritization, and fully defer to @ask for that.

[image] PoolMUC: Also, some other technical and non-technical questions need to be considered first. E.g., as mentioned, other zones have smaller servers operating without issue because of a “good” client to server ratio in those zones. It needs to be managed carefully not to overload them, as could likely happen if the load of some larger zones is suddenly hitting the wider population with the flip of a switch. A more gradual approach is needed, e.g., by more and more people over time switching to the global zone. Or the infrastructure having a mechanism to slowly ramp up the share of traffic distributed outside the originating zone over time. And @ask has some plans on it. [image] Fill zero-server pools with CNAME The DNS system makes logs (and metrics) for how many DNS queries it gets from each country. Obviously that doesn’t quite correspond to NTP queries, but it should be a good proxy. My plan is to calculate for each country with less than X servers (or X “net speed capacity”) how far off it is the median/average/some reasonable comparison country and then supplement with servers either from the global zone or that have explicitly joined some other “extra global” zone. (If we use the global zone I’l…

[image] Coelacanthus: And @ask has some plans on it. Yes, those are what I keep referring to as technical solution, and there are different aspects being discussed in various other threads over time since then, but it is unclear to me as to where we stand with those plans as of today.

When the pool provides .pool.ntp.org domains, users naturally intend to use one that matches their regions. Only those with some insights on how NTP and the NTP pool works may think differ. So for majority of users that having “configuring a NTP server” in their minds, using their regional pool is a de facto standard, and then many tutorials and documentations are saying so. Clearly there’s not much we can do to change what domain users chose, not to mention most clients are not configured by the end users, but by product vendors. I agree that using the global pool for all regions is a good approach, and that could/should be easily implemented on the DNS layer: just CNAME all regional zones to the global one. ps, I’m the one added some far away servers to pool recently and been refused to add them to the CN zone due to physical distance, I’d be more than happy to see a feature allowing server operators to set zones of their servers in the manage panel.

[image] HAHAHAHA: need to promote the use of global pool Another thing coming to mind in this context: For now, it should also be promoted to use the zones prefixed with “2.”, as only those have IPv6 servers in them. Those are often underutilized as compared to the IPv4 servers, so using the “2.” zones also helps offload a bit of traffic from the IPv4 servers. (See various other threads in this forum for background discussion on that topic.)

In fact, I don’t know whether it’s necessary to have 0/1/2/3 in front of the global pool. Why not just pool.ntp.org and have both Ipv6 and Ipv4 enabled for it?

If regional pools are eliminated, user cannot use them so pool.ntp.org will be the default.

CN pool collapse a few hours every day

Server operators

Topic		Replies	Views
Adding servers to the China zone Server operators	384	29994	June 9, 2022
Gradually add/remove server to/from pool in parallel to score increase/decrease Pool Development monitoring , dns	78	3303	August 22, 2024
Collapse of Russia country zone Server operators	202	6004	December 9, 2024
Many (probably all) NTP servers in the Philippines don't work Server operators	56	2726	September 5, 2026
The issue of NTP requests exceeding bandwidth load Server operators	54	1858	November 24, 2024

CN pool collapse a few hours every day

Related topics