This might have been answered before, so please excuse the dumb question. A quick forum search didn’t have any answers.
It has been raised in other threads that if there is even only one server in a zone then all traffic for that zone will hit that server, instead of looking around for other less loaded zones.
But then why is US, with 1000 servers hitting servers in Africa where there is only 100 servers for the whole continent? I server Africa, and US is my 4’th biggest client, and the only non Africa client?
And I have seen US pop up in Europe servers too, as the only non Europe zone client?
With respect to your question, there are two types of zones: named and unnamed.
Named zones have a country code or (sub-)continent in their name. Regardless of from where they are queried, one gets servers from that zone. Or none, if there aren’t any.
Unnamed zones are sometimes also called global zones, though I find the latter a bit misleading. As they do not give servers from around the globe, but they are simply generically valid wherever the client is, and one then gets servers from the local zone, as best as the pool can determine a client’s location. And those zones dohave a fallback: At long as there is at least one server in the zone, that one is returned. If there are none, servers from the surrounding zone are provided, e.g., from the continent zone when there aren’t any servers in the local country zone.
As to how clients from other zones end up on a server, there’s multiple reasons. E.g., people in an underserved zone might configure a specific foreign country zone to get better service. Or for backup purposes, though it depends on circumstances as to whether that makes sense.
Also, just as servers might be misplaced by the pool, so can be clients. And MaxMind, on whose data the zone determination is based, does not care where the actual IP address is located, but where the user of an address is. E.g., when a previous user of an IP address was located in another country, then it’s hard to get them to correct that data, and the pool will serve the client with addresses from the “wrong” zone.
Servers configured for @ zone serve requests from all over the world, including US. To remove this you should lower your server’s speed setting to the lower settings < 12Mbps.
I know this is a widespread assumption, and maybe even the descriptions on the web site suggest this kind of behavior. However, based on my looking into these topics, starting from/confirming other people’s findings, I don’t think that is how it actually works.
What is definitely true, though, is that when a client is configured with an unnamed zone, and there are no servers in that zone, so that the client gets handed servers from the enclosing zone as fall-back, that obviously means that those servers in other zones will see this as a request coming from outside their own respective local zone.
Honestly, I’ve always wondered about the “@” zone, and how it works, i.e., under which circumstances a client would be handed a server really from somewhere across the globe. The only thing I could come up with is that should a continent zone also not have any servers in it, the fallback would then be to the global “@” zone. But as there are few continent zones that don’t have any servers in them, this is hard to verify. And the flights to Antarctica were sold out before I could get a ticket to test from there …
Another question on my mind is how the netspeed setting is applied when a fall-back is active. It mostly makes sense within a country zone, and as discussed variously in this forum, netspeed settings cannot be compared across countries, i.e., same netspeed in two countries can result in widely varying actual traffic.
So how does that square when servers from different country zones are to be fed into a single DNS response? Only thing I can think of is that even then, the netspeed is applied as is, comparing against the netspeeds of servers in other zones, but that this is not really that relevant in this case because the load generated that way is perhaps negligible in comparison to the load generated by a local zone. I.e., the netspeed controls primarily the traffic from the local zone, and that it might not be “accurate” to compare servers across zones doesn’t matter because there is too little traffic from that side to actually make a difference. Or it is factored in, i.e., it is anyhow not possible to universally say “netspeed setting X will generate traffic load Y” across all zones, but it is specific to each zone. Thus, operators anyhow need to experiment with the setting and find the “right” one for themselves, and that process implicitly takes into account also traffic coming from other zones.
Actually, just to be precise, the actual threshold is < 10Mbit. So when a server is added to the pool for the first time, based on its default 10Mbit setting, it will be included in the “@” zone. But once the netspeed has been changed through the UI to anything else, 12Mbit becomes the de facto threshold, because the next lower value below 10Mbit available via the UI is 6Mbit.
Please also note that it is not Mbps or Kbps (i.e., per second), but rather Mbit or Kbit. And even that may be confusing, as this value has no direct and generally valid correlation to the actual amount of traffic that the server will receive. It will be a very close match for some zones, but for many, if not most, it is not.
Forgot one: Use of privacy-minded anycast DNS resolver services such as Quad9 and Cloudflare.
With those, the pool is not able to detect the location of the client directly (because the DNS resolver hides relevant information from the pool’s DNS servers). Rather, the pool then takes the location of the DNS server as the zone for which to provide NTP servers to the requesting client. And if the specific DNS server instance forwarding the request to the pool’s authoritative nameservers happens to be in another country, that is what the pool will assume as the client’s zone.
There was a nice presentation at one of the last RIPE meetings that nicely illustrates the “watersheds” for anycast DNS resolver services. The figures are rather coarse, but I think one can roughly see which geographic areas are served by which anycast DNS instances, and that in various cases, regions are served by instances in other geographies. E.g., because the region/country doesn’t host an anycast instance. Or just because the routing in the Internet, at that time, happened to send requests from one place to an anycast instance in another country, even though the origin country did have its own anycast instance.
If the client is in US, but uses a privacy DNS server in, say, Africa, then that would still, from the point of view of the stats, look like a client from Africa, would it not?
And if MaxMind is used in the stats and the DNS server finding closest IPs, then incorrect MaxMind data would also “balance out” and, even if the client is in US but MaxMind thinks he is in Africa, he would get an IP in Africa, but he would also be reported as an African client?
Ok, so my bandwidth is set at 3Gbit. And I’m part of @. When does a client get assigned an IP outside their zone when they do have a server in their zone?
The only thing I could come up with is that should a continent zone also not have any servers in it, the fallback would then be to the global “@” zone.
But the US itself has many servers, so this also does not work?
I’m just trying to match this up with the issue Bas is having(had?) problems with an overloaded zone. In his first post( I take a big hit in queries ) his server is 36X overloaded in his own zone, and US is also 3’rd in his clients?
Yes. Hmm, but I think I get a feeling for what you are after. Need to think about this a bit more. I think maybe it is different for different perspectives, and maybe my thinking assumed one thing for both, which is actually not valid for one of the perspectives. (When does a client get assigned a server outside its own actual zone, vs. how do clients end up at a server so that the server counts them as coming from another zone.)
Not yet sure what you mean by “balance out”.
I assume by “get assigned an IP outside their zone”, you mean the IP address of a server outside the client’s own zone. I.e., this is the “client perspective”. Then, two options come to mind:
When the client is configured with a named zone which is not its own.
When the pool thinks (for one of multiple possible reasons) the client is in another zone than it actually is.
But that client would only show up as out-of-zone from “server perspective” (as in being counted by the server as out-of-zone) in the first of these cases, but not the second. Maybe that is what you mean by “balance out”?
One path where the second item above might still be relevant for a client popping up as out-of-zone for a country is when the user might notice that the client is placed in the wrong place, as it is getting assigned servers “too far away”. For those “far-away” servers, it might in that case pop up as from the server’s own zone. However, this might drive the user to try to manually override the pool-side selection by explicitly configuring their client to use the local zone.
I.e., this would kind of turn the second item above (where the client doesn’t pop up as out-of-zone for any server) into the first item above, where it will be counted as out-of-zone by the local server.
Yes, this is indeed strange, certainly in the magnitude in which it seems to be happening at least in your case, and I don’t have a good explanation for this right now. The above are just some things that can happen for clients from other zones to pop up at a server in a specific zone (and one of them actually isn’t even directly relevant in this context, as I think I might just have realized, because the server does not actually see/count it as out-of-zone).
One hypothesis could be that for various reasons, more addresses are misplaced to be in the USA than, e.g., addresses from the USA to be misplaced somewhere in Africa. E.g., not sure how Starlink customers are geo-located. If in doubt, they might likely/possibly end up being placed in the USA, triggering at least some users to manually select the local pool zone (similar for various other areas of IT dominated by tech companies from the USA).
The thing I can’t square right now is that from all I have seen so far, I currently don’t know what the mechanism/circumstances would be where a client would be assigned a server from a zone on an entirely different continent while still properly being counted by the server as from out-of-zone, except when explicitly configuring that remote zone.
I.e., can anyone describe a scenario where that would happen, i.e., where clients would really actually be assigned to servers on a different continent, while both server and client are properly geo-located in different countries on different continents, and without the client having configured the remote zone? I.e., where really the servers would be drawn from the “@” zone containing all servers world-wide with a netspeed setting of 10Mbit or above? @alica?
I don’t think the number of out-of-zone clients had much bearing on the overload situation he was experiencing. While visible in the stats, and probably visible in the actual traffic as well, I guess it still was not the decisive cause of the overload. In the sense that without this traffic, the change would likely not have been completely binary/black-and-white. But digging deeper into the numbers might also prove this guess wrong.
It is not per se “overloaded”. The number itself basically just means that there for some reason is a very noticeable disproportion between the share that the configured netspeed value has from the overall sum of netspeeds of all servers, and how the DNS load-sharing translates that into how often the IP address is included in responses to DNS queries from his own zone.
Just that it is third by itself doesn’t necessarily mean anything. One would have to look at the actual, absolute numbers that lead to that ranking. I also would need to re-read the entire other thread to see whether Bas himself demonstrated as least in some way how that foreign traffic was actually contributing to the overload situation, vs. experiencing an overload situation, then looking at the traffic, seeing “some” out-of-zone traffic, and then attributing the overload to out-of-zone traffic, or just generally complaining that he doesn’t want to see foreign traffic on his server as a matter of principle (as said, I don’t claim/remember he did either or any of those, would need to re-read the entire thread first).
With “balance out” I meant that if the DNS software uses maxmind, and thinks that the client is in zone CI, and the stats software uses maxmind then the stat software will also think that the client is in zone CI, so he will show in the stats as zone CI, even if he was in zone US all along. What I tried to say in a long-winded way was that maxmind should not be causing this miss-allocation. Don’t get me wrong, it probably does a lot of miss-allocation, but the stats should then also miss-allocate that client.
No, Bas was not complaining about the foreign traffic at all, I,m just using him as another example of high foreign traffic. This just struck me as interesting from his thread:
Same zone only. Even if global pool is used.
According to the stats page, the served zones are sorted according to the absolute number of queries, so I’m serving a small percentage of the total US queries, but in actual request number of queries it is still more than some of the actual request numbers for zones in my own continent.
So the only conclusion I can come to is that there are users in the US that use the africa.pool.ntp.org pool. Don’t know why though… People that moved from Africa to the US and never updated their device’s pool settings?
But I don’t think there is an answer to this questions.
I considered that, but why doesn’t other zone’s monitors then also show up?
Also, monitors are “assigned” to servers, so have a set list of IPs that they monitor, they don’t just pull an IP from the pool? So they should not show up in the DNS stats…
Yes, the DNS stats shown on a server’s page are drawn from what the GeoDNS system reports. The monitoring system does not use that GeoDNS system to allocate monitors to servers (unlike when actual clients are directed to servers), so the monitors’ traffic does not show up in the GeoDNS system’s stats.
Not yet at least, let’s see whether further pondering, prodding, and poking around will eventually lift the mystery At least I am curious to learn how this happens, purely out of curiosity.
The NTP-monitors are just that, they monitor NTP-servers around the world and they are divided in different groups based on their reputation to monitor e.g. your server.
Their only purpose is to examine if your servers is ticking time properly and is reachable.
Other then that they have NO function. If you qualify, your server is put in the pool….
However, this pool is not zone, global etc…it shows as such, but from what I understand it doesn’t work that way.
I believe every country is serving it’s own users UNTIL there are no servers anymore, then Zone and Global will be asked to help, probably Global first.
If a country has NO servers, it seeks help from other Zone/Global servers to give time.
The problem is, when a country has servers, but not enough, it gets almost ALL users dropped on this server and it overloads.
It may have changed by now, hope so, but I asked HELP from others to help serving Belgium, some BIG GUNS stepped in and my load dropped to normal levels.
The %-numbers mean nothing, only when they go higher your server gets more traffic…but other then that? No clue.
In short, if your server is overloaded, ask help, plenty others willing to serve your country too! So the load goes down.
That is the short solution until @ask fixes the DNS-round-robin system, so Zone and Global servers step in much sooner.
Beware, this is what I believe it does, and what people told me to do etc.
My question relates to “why do I, in Africa, serve USA when the USA has 600 of their own servers, and Africa, as a continent, only has 67 servers?”, so none of the “if they have no servers then bla bla bla” arguments works… I have seen that you @Bas also serves USA from Europe.
The order of the countries listed on the server’s stats page is significant. It is ordered according to the actual number of DNS replies that included my server’s IP, so if US is above UG then it means that my server’s IP was returned to more US(North America) clients than to UG(Africa) clients.
The monitors will NOT show up in the DNS stats, as they DONT use the DNS pool to get IPs.
As with other topics in the past, you seem prone to invalid generalizations: Just because you don’t understand the numbers, or rather, don’t seem to find them useful, does not mean that everyone else (“we”) also does not know what they mean, or does not find them useful.
I know what they mean (to me), I am fully aware of their limitations, and what they do not mean, and within that scope find them useful. And I am sure there are others for whom this is true as well.
At least I am curious to learn how this happens, purely out of curiosity.