Also experiencing this on my server.
Anything i can try?
I was having similar issues, then the other day I happened to run ‘ntpq -p’ on my server just to see the stats and was shocked that some of the servers were way off. I’m guessing network issues somewhere. But I guess NTP was jumping around not really sure which group was the correct time, and when it did my score would drop on the monitoring page.
I had to go through quite a few servers before I got enough that agreed on the time and were stable. Once I solved that problem my score has been steadily going back up on the monitoring page fingers crossed.
I realize it’s a long shot, but if you don’t watch the ntp server list stats regularly, it might be worth checking on.
Following up on my previous posts for 126.96.36.199
I suppose it is some consolation that it seems quite a few others are seeing similar issues… Anyway, I found some improvement after disabling ‘Port Scan and DoS Protection’ on my router, but didn’t solve it completely. As my scores now do occasionally get above 10 I thought I would put it back in the Beta Pool - and, although I haven’t reviewed a large amount of history, I can see that the timeouts occur just on the LA monitoring system but Zurich seems to be fine. Ah - having said that, I’ve just now seen one i/o timeout for Zurich, but it doesn’t change the point that Zurich is much better.
I am left wondering if the difference is related to the routing to the two monitoring locations or to do with the fact that the Zurich one does three samples.
Hi thanks for the reply, i have tried changing my peers although it does not seem to change much. I think it must be a network issue between me and the monitoring station. I wish it was slightly more forgiving.
Still experiencing the issue. After looking at the logs it looks like its the io/timeout error everyone else is experiencing. https://www.ntppool.org/scores/188.8.131.52/log?limit=200&monitor=* Not sure what to do really. Does anyone know the current IP of the monitor?
It looks like scoring system really suffers from being located only in US.
I have listed both ipv4 and ipv6 of the same server (hosted at Scaleway @ Netherlands), and they are getting drastically different scores, with ipv6 being much better (!!!???). Currently ipv6 has score of 6.5, and ipv4 has score of -19.2.
Servers are 184.108.40.206 and 2001:bc8:4730:940e::1
We are seeing this be an issue as well, and I think it is because of the monitoring station being US based.
We have an NTP server running on a dedicated 100mb/100mb fibre loop. Using the NTP Server Test at https://servertest.online/ntp we consistently get good results, but from the NTP Pool monitoring station we are getting scores of -88.6!! I cannot fathom why. The CSV (https://www.ntppool.org/scores/220.127.116.11/log?limit=200&monitor=*) shows I/O timeouts but I don’t see why - the server is up, accepting connections and available. There is no traffic being put down the connection (apart from NTP), so the only conclusion I can draw is that the network between us and the monitoring station is providing an extreme/excessive amount of latency. Querying the NTP server from within the UK, where it is based, returns very quick responses and we certainly aren’t seeing I/O timeouts.
This is a real shame as we would like to provide resource to benefit the NTP Pool project. As far as we can see we have done everything right, it’s just factors outside of our control that seem to be stumping us.
Well, there seems to be something dropping the traffic. I don’t think this is only a NTP Pool monitor node issue. Stats from Finland: http://orava.miuku.net/stats/ntppacketloss-18.104.22.168.html
Thanks for the reply - that is very, very interesting… Nothing in our network has been found to be dropping NTP packets. My only guess is that it is upstream from us at our ISP
Months and months later IPv4 scores still dive and rise. Ipv6 holds stable.
Indeed! My server is still marked ‘rubbish’ most of the time. The beta system only shows LA now, so I can’t see the comparison with Zurich anymore either…
Another NTP server on Scaleway here.
I’ve done a packet capture and found that my server is correctly receiving requests and generating replies.
Traceroute to 22.214.171.124 1 gw-b.develooper.com (126.96.36.199) AS7012 0.246 0.199 2 gi1-9.r01.lax2.phyber.com (188.8.131.52) AS7012 0.959 1.041 3 te0-1-0-7.r04.lax02.as7012.net (184.108.40.206) AS7012 0.878 1.042 4 te0-10-0-20.ccr41.lax04.atlas.cogentco.com (220.127.116.11) AS174 0.934 1.026 5 (18.104.22.168) AS174 0.975 5 be3360.ccr42.lax01.atlas.cogentco.com (22.214.171.124) AS174 0.785 6 be2932.ccr32.phx01.atlas.cogentco.com (126.96.36.199) AS174 12.690 12.856 7 be2929.ccr21.elp01.atlas.cogentco.com (188.8.131.52) AS174 20.730 20.718 8 (184.108.40.206) AS174 36.619 8 be2927.ccr41.iah01.atlas.cogentco.com (220.127.116.11) AS174 36.545 9 (18.104.22.168) AS174 50.894 9 be2687.ccr41.atl01.atlas.cogentco.com (22.214.171.124) AS174 51.077 10 be2112.ccr41.dca01.atlas.cogentco.com (126.96.36.199) AS174 61.901 61.741 11 be2807.ccr42.jfk02.atlas.cogentco.com (188.8.131.52) AS174 68.058 68.051 12 be2317.ccr41.lon13.atlas.cogentco.com (184.108.40.206) AS174 138.701 138.593 13 (220.127.116.11) AS174 145.275 13 be12194.ccr41.ams03.atlas.cogentco.com (18.104.22.168) AS174 144.952 14 be3434.rcr21.ams06.atlas.cogentco.com (22.214.171.124) AS174 146.587 146.624 15 online.demarc.cogentco.com (126.96.36.199) AS174 152.821 152.800 16 (188.8.131.52) AS12876 153.572 16 (184.108.40.206) AS12876 147.730 17 * * 18 * * 19 * * 20 * * 21 quicksilver.scw-ams.ifcfg.info (220.127.116.11) AS12876 152.322 154.670
Update: I came across this post and now the issue seems really strange. The author has two servers both in the same IP address block (and I believe they are both Scaleway?), and one of them has a bad score just like the rest of us, but the other one does not seem to be affected at all.
Update again: Traceroute to the “good” server seems to be completely different:
Traceroute to 18.104.22.168 1 gw-b.develooper.com (22.214.171.124) AS7012 0.242 0.166 2 gi1-9.r01.lax2.phyber.com (126.96.36.199) AS7012 0.644 0.606 3 te0-1-0-7.r04.lax02.as7012.net (188.8.131.52) AS7012 0.993 3.441 4 xe-0-1-0-30.r01.lsanca07.us.bb.gin.ntt.net (184.108.40.206) AS2914 0.688 0.676 5 ae-19.r00.lsanca07.us.bb.gin.ntt.net (220.127.116.11) AS2914 1.090 0.826 6 ae-0.level3.lsanca07.us.bb.gin.ntt.net (18.104.22.168) AS2914 0.764 0.907 7 * * 8 (22.214.171.124) AS3356 141.116 141.111 9 (126.96.36.199) AS12876 141.286 9 (188.8.131.52) AS12876 140.387 10 * * 11 * * 12 * * 13 * * 14 * * 15 * * 16 * * 17 * * 18 * * 19 * * 20 * * 21 * * 22 * * 23 * * 24 * * 25 * * 26 * * 27 * * 28 * * 29 * * 30 * *
Traceroute to the “bad” one, however, looks the same as that to my server.
the mysterious thing is that both have ipv6 connectivity and the score is about 18.x
The “good” one lives in DC Paris
The “bad” one lives in DC Amsterdam
ps: i’m the author from the other thread
pps: the bad one is gone. I setup a new one in Paris - maybe this one is better
So it looks like routing problem between Scaleway DC AMS and LA monitor station now.
Maybe just the monitoring station in general. My New Jersey VPS in the USA and my VPS in Hong Kong both dip and dive.
Scaleway used to have an issue on part of their network that corrupted the checksums on about 50% of outbound UDP packets. No joke.
For the person I heard about it from, Scaleway fixed it around September, but maybe other parts of their network are still affected.
I have done some test to my server from outside Scaleway, mostly NL/HK/US. In most cases there is no problem. Although on some of the test servers there are a few packets lost, it is not like the continued “i/o timeout” the monitor has seen.
An interesting thing is that I did not get any reply on one particular server in US, and traceroute to my NTP server on that machine shows almost the same route as from the monitor.
I am seeing this same behavior on a Digital Ocean server in NYC. It happened a few months back, and has now happened at least twice in the past 2-3 days. I joined the beta server pool and it’s happy as a clam. over a time period that the LA monitoring server was scoring my server around 0. Everything about the hosting network looks good to me, as does my server.
My NYC1 droplet’s IPv4 address’s score went as low as -76 yesterday. (It’s more or less okay now.)
I was able to confirm that the issue didn’t just affect the monitoring server. (But I didn’t entirely track it down, or report it to DigitalOcean.)
Edit: It has two IPv4 addresses in the beta pool. The non-floating IP maintained a perfect score.
My FR dedicated server with online.net (by scaleway) seems to have similar issues. Since it has the same configuration as my hetzner servers (which mostly keeps at score 20), I believe it’s hardly a configuration issue from my side.
It looks better than the above charts from scaleway VPS though.