Based on https://community.ntppool.org/t/stratum-pool-servers/1529/7 I thought I would ask the question 
Is there an “easy” way, other than brute force* to, on an ongoing basis, find pool servers that have a score between 5 and 15? (Following down the path of improving the quality of the pool)
Brute force = Do lots of DNS queries to find the pool server ip addresses then query the scores api endpoint to see and record what the current score is for each ip.
I can’t say I’d suggest going this route, but in addition to making tons of DNS queries it’s technically possible to use the server’s numeric ID. For example, https://www.ntppool.org/s/64166 redirects to https://www.ntppool.org/scores/45.115.225.48. You may have seen such shorter URLs in “Problems with your NTP server” emails.
This is still a fairly brute force method. On the other hand, this method would also find servers with a score <10, which you wouldn’t find via DNS queries.
@gombadi The scoring data still gets exported to Google BigQuery; and it’s also stored in avro files (though I don’t think they are accessible anymore; nobody was using them so I didn’t setup access again in one of the migrations over the years).
If you can tell what you are trying to do and it’s related to making the system better then I’m happy to help provide more access to the data.
Over the past while I have been playing around with a pool monitoring system. It gathers results that I have a look at. The results are public if anyone wants to have a look. This is a summary page and the format/data can change at any time though so you should not rely on the page staying the same.
While looking at the results I saw that some servers were always showing up in some results. Doing a manual score check of them I saw that quite a few had scores that never seemed (I only had snapshots in time) to reach 20 or even go above 15. They would quite often bounce around going in and out of the pool. For example https://www.ntppool.org/scores/23.106.249.200 has a score of 12.2 with most monitors detecting a clock offset of about 80mSec. Dear server owner can we help in improving the time that your server is giving out to users. Also https://www.ntppool.org/scores/43.229.16.158 currently with a score of 18.1 but looking at the test results for 20th Sept -
$ zcat *.gz | grep -v Tested_CAC | grep 43.229.16.158 | awk -F, '{print $2}' | sort | uniq -c | sort -rn
52 Tested_ERR
42 Tested_OK
More tests failed than worked. Can we work with the server owner to improve the performance of the server.
Instead of trying to monitor/check all the servers in the pool I thought it would be better to focus on servers with scores mostly between 5 and 15. I could make numerous requests to the pool api to check a servers current score but that seemed to much of a brute force attack so started this thread to see if there was a better way. There may be
That is pretty much what I am doing. Happy to answer any questions anyone may have.
The “constantly 80ms off” is pretty amazing. I wonder, too, how often that happens.
I’m curious too what the impact would be if we just tightened the accuracy requirement (I already did sometime in the last year or two); or if we should have some special logic where a server get a lower score if the time is off and not getting better.
Another area of inquiry would be what signals (all monitors saying “nope” in a particular way, for example) are strong signals that a server should be removed from the pool faster.
It looks like 23.106.249.200 has been advertising stratum 3, with a reference ID of 127.0.0.1 all year. Typically 127.0.0.1 means that the server is syncing to an internal clock. The root delay is 240 msec, which is huge. Last year the same server had a reference ID of 130.133.1.10 time.fu-berlin.de, at least part of the time, with a root delay of 220 msec.
My guess is that the operator has overridden the reference ID for some reason, and is still syncing halfway around the world. If true, its no wonder that there is an 80 msec offset.
There is quite a bit of asymmetry on IP networks. It can be difficult to distinguish server offset from monitor-server asymmetry.
Doesn’t have to be the case. As then it would list as Stratum 1 not 3.
It’s probably using the internal clocksource but synching with stratum 2 servers.
Because it’s on www.leaseweb.com, I suspect it to be a VPS, maybe on a host at very low priority.
I have seen many hosters to oversell virtual-machines and they often have very poor times, too little CPU-cycles to do proper work.
My 2cts.