Issues with China regional servers

Server operators
1

Since I am not proficient in English, the machine-translated text might sound awkward.

Servers deployed in China are intended to serve devices within the Chinese mainland. Local telemetry indicates a latency of only about 12 milliseconds and negligible time offset. However, the server’s lifecycle and scoring are heavily influenced by transoceanic monitoring nodes located in Europe and the Americas. Any jitter in trans-Pacific routing or packet loss on international links results in severe score penalties, causing the rating to drop below the 10.0-point threshold. Excluding these otherwise healthy local nodes fails to protect the NTP resource pool; instead, it places an excessive burden on other Chinese servers, creating a vicious cycle. I believe that for servers in China, scoring should prioritize latency and reachability within the target geographic region, rather than treating packet loss detected by monitoring nodes thousands of kilometers away as a critical flaw (especially given the scarcity of servers in the region).

2

Hi and welcome to the pool.

Are you able to share the ip addresses of the pool servers you are having problems with?

I have a server running in Hong Kong and the pool monitoring page https://www.ntppool.org/scores/38.60.227.120 shows the monitoring servers that are checking it. All the active monitors are in Asia - Hong Kong, Taiwan and Vietnam - so no trans continent checks are involved. If we can see the ip address and/or the pool monitoring link for the China based servers you are having problems with we may be able to help in sorting out the problem.

3

I randomly picked a cn server and check its status page:

As you may notice, some of the nearby monitors (namely hkhkg1-3strqkc and sgsin1-z4ytm9) can not access the server at all, some others (like twtsa1-11vrqqm and twtpe2-2kg9ezv) have RTT in near 200ms, almost in a trans-continent level. There is no guarantee monitors outside GFW can always reach servers inside GFW; we still need more monitors inside GFW to better evaluate servers inside GFW.

4

This has been raised several times in the past already, and not only with respect to the CN zone. In response to such requests (among other things), the monitoring system has been enhanced so that it is easier for people with a track record of hosting a server to also host monitors. And it worked, there are now well over 150 monitors for IPv4 from all over the place.

However, for some reason, out of those well over 150 monitors, there is only a single one in the CN zone. I’ve wondered before why that is.

So you guys in the CN zone need to figure out amongst yourselves why there are not more monitors in the CN zone. Setting one up is easy. The prerequisite for hosting a monitor (having had a server verified for at least 18 months) can be waived for cases where this makes sense, and where the pool would benefit from more monitors to increase diversity (send an e-mail to server-owner-help@ntppool.org).

For the same reason that you rightly write that a bad score from a monitor far away is not necessarily indicative of how the server performs for more local clients does it not matter when a monitor is giving bad scores to servers far away. Even any issues in between ISPs within the CN zone should not be a deterrent not to run a monitor within the CN zone. I assume even regional ISPs within the CN zone have enough customers in them to be worthwhile to be represented by a monitor within that ISP network. Only the uplink from the monitor needs to be stable, not so much the inner workings of the network farther away, as the latter is precisely what a monitor is supposed to assess.

One challenge is that the monitoring system itself is biased in the sense that the health of the monitors themselves is evaluated against a fixed set of reference servers. And those are currently located so as to disfavor monitors in many places in the world. E.g., there is a single monitor in India (another place that would benefit from additional monitors), and it is not monitoring most of the time because it is sanity-checked against reference servers “far away”, and due to that is considered having a bad clock most of the time when it would be ok with respect to more local references. That bias is another problem, but again it should not be a reason not to run a monitor. Only when we learn about such issues, and they are exposed, can there be improvement in those areas, at least potentially.

So please communicate within the local communities that the pool not only needs more servers in the CN zone, but more monitors as well. Reach out to universities and companies, and invite them to host monitors for the pool (and servers).

And, if you have a stable uplink, host a monitor yourself.

5
6

Thank you for such a transparent and constructive explanation.
Regarding your suggestion to “host a monitor yourself” if one has a stable uplink, I worry we might end up following the same path as the Indian monitor—falling into a perpetual “not monitoring” state due to packet loss when communicating with distantservers. I believe we need to come up with a solution for this.

7

Yes, but as I tried to motivate, this should not be a reason not to host a monitor. Otherwise, nothing will ever change until you get a perfect Internet, which will be never. So again the encouragement to run a monitor despite the likelihood of such issues. The point of the monitor is to reflect what local clients would see, not what far-away clients see.

And monitors in the CN zone, and the one in the IN zone, are not alone. Many other places are affected as well, to varying degrees. E.g., BR, RU, MY, SG, ID, MD, AU, to name a few where I have previously observed the same issue.

Sure, but if we wait until the perfect solution is found, nothing will ever change. Baby steps are key. And the first is to host monitors in places currently disadvantaged by the current choice of references for the monitors themselves.

Given how many monitors there are by now, and how the scoring works, that it always picks the best-scoring monitors for a server, regardless of where they are, that almost renders the separate checking of the monitors obsolete. I.e., the requirements for them could be relaxed to only cover a rough sanity check, and the scoring will be the actual per-server quality indicator. Just as having far-away monitors monitor servers doesn’t make sense, so does checking monitors against far-away references not make sense, and is counter-productive.

8

Okay, I now have a server that I think has a good network. If possible, I can try to let it participate in network monitoring.

9

That would be really great! Not to get discouraged by what could happen, but just giving it a try. Indeed your server looks good, so just give it a try.