Score/network woes

monitoring

#54

Hi All,

Also experiencing this on my server.

Anything i can try?

Thanks


#55

I was having similar issues, then the other day I happened to run ‘ntpq -p’ on my server just to see the stats and was shocked that some of the servers were way off. I’m guessing network issues somewhere. But I guess NTP was jumping around not really sure which group was the correct time, and when it did my score would drop on the monitoring page.

I had to go through quite a few servers before I got enough that agreed on the time and were stable. Once I solved that problem my score has been steadily going back up on the monitoring page fingers crossed.

I realize it’s a long shot, but if you don’t watch the ntp server list stats regularly, it might be worth checking on.


#56

Following up on my previous posts for 81.174.128.183
I suppose it is some consolation that it seems quite a few others are seeing similar issues… Anyway, I found some improvement after disabling ‘Port Scan and DoS Protection’ on my router, but didn’t solve it completely. As my scores now do occasionally get above 10 I thought I would put it back in the Beta Pool - and, although I haven’t reviewed a large amount of history, I can see that the timeouts occur just on the LA monitoring system but Zurich seems to be fine. Ah - having said that, I’ve just now seen one i/o timeout for Zurich, but it doesn’t change the point that Zurich is much better.
I am left wondering if the difference is related to the routing to the two monitoring locations or to do with the fact that the Zurich one does three samples.


#57

Hi thanks for the reply, i have tried changing my peers although it does not seem to change much. I think it must be a network issue between me and the monitoring station. I wish it was slightly more forgiving.


#58

Still experiencing the issue. After looking at the logs it looks like its the io/timeout error everyone else is experiencing. https://www.ntppool.org/scores/81.174.133.68/log?limit=200&monitor=* Not sure what to do really. Does anyone know the current IP of the monitor?


#59

It looks like scoring system really suffers from being located only in US.

I have listed both ipv4 and ipv6 of the same server (hosted at Scaleway @ Netherlands), and they are getting drastically different scores, with ipv6 being much better (!!!???). Currently ipv6 has score of 6.5, and ipv4 has score of -19.2.

Servers are 51.15.74.121 and 2001:bc8:4730:940e::1


#60

We are seeing this be an issue as well, and I think it is because of the monitoring station being US based.

We have an NTP server running on a dedicated 100mb/100mb fibre loop. Using the NTP Server Test at https://servertest.online/ntp we consistently get good results, but from the NTP Pool monitoring station we are getting scores of -88.6!! I cannot fathom why. The CSV (https://www.ntppool.org/scores/62.254.125.171/log?limit=200&monitor=*) shows I/O timeouts but I don’t see why - the server is up, accepting connections and available. There is no traffic being put down the connection (apart from NTP), so the only conclusion I can draw is that the network between us and the monitoring station is providing an extreme/excessive amount of latency. Querying the NTP server from within the UK, where it is based, returns very quick responses and we certainly aren’t seeing I/O timeouts.

This is a real shame as we would like to provide resource to benefit the NTP Pool project. As far as we can see we have done everything right, it’s just factors outside of our control that seem to be stumping us.


#61

Well, there seems to be something dropping the traffic. I don’t think this is only a NTP Pool monitor node issue. Stats from Finland: http://orava.miuku.net/stats/ntppacketloss-62.254.125.171.html


#62

Thanks for the reply - that is very, very interesting… Nothing in our network has been found to be dropping NTP packets. My only guess is that it is upstream from us at our ISP :persevere:


#63

Months and months later IPv4 scores still dive and rise. Ipv6 holds stable.


#64

Indeed! My server is still marked ‘rubbish’ most of the time. The beta system only shows LA now, so I can’t see the comparison with Zurich anymore either…


#65

Another NTP server on Scaleway here.

I’ve done a packet capture and found that my server is correctly receiving requests and generating replies.

Traceroute:

Summary
Traceroute to 51.15.92.0
 1 gw-b.develooper.com (207.171.7.3) AS7012  0.246  0.199
 2 gi1-9.r01.lax2.phyber.com (207.171.30.13) AS7012  0.959  1.041
 3 te0-1-0-7.r04.lax02.as7012.net (207.171.30.61) AS7012  0.878  1.042
 4 te0-10-0-20.ccr41.lax04.atlas.cogentco.com (38.88.197.81) AS174  0.934  1.026
 5 (154.54.42.101) AS174  0.975
 5 be3360.ccr42.lax01.atlas.cogentco.com (154.54.25.149) AS174  0.785
 6 be2932.ccr32.phx01.atlas.cogentco.com (154.54.45.161) AS174  12.690  12.856
 7 be2929.ccr21.elp01.atlas.cogentco.com (154.54.42.66) AS174  20.730  20.718
 8 (154.54.30.161) AS174  36.619
 8 be2927.ccr41.iah01.atlas.cogentco.com (154.54.29.221) AS174  36.545
 9 (154.54.28.129) AS174  50.894
 9 be2687.ccr41.atl01.atlas.cogentco.com (154.54.28.69) AS174  51.077
10 be2112.ccr41.dca01.atlas.cogentco.com (154.54.7.157) AS174  61.901  61.741
11 be2807.ccr42.jfk02.atlas.cogentco.com (154.54.40.109) AS174  68.058  68.051
12 be2317.ccr41.lon13.atlas.cogentco.com (154.54.30.186) AS174  138.701  138.593
13 (130.117.51.42) AS174  145.275
13 be12194.ccr41.ams03.atlas.cogentco.com (154.54.56.94) AS174  144.952
14 be3434.rcr21.ams06.atlas.cogentco.com (154.54.59.50) AS174  146.587  146.624
15 online.demarc.cogentco.com (149.14.92.210) AS174  152.821  152.800
16 (51.158.8.25) AS12876  153.572
16 (51.158.8.168) AS12876  147.730
17  *  *
18  *  *
19  *  *
20  *  *
21 quicksilver.scw-ams.ifcfg.info (51.15.92.0) AS12876  152.322  154.670

Update: I came across this post and now the issue seems really strange. The author has two servers both in the same IP address block (and I believe they are both Scaleway?), and one of them has a bad score just like the rest of us, but the other one does not seem to be affected at all.

Update again: Traceroute to the “good” server seems to be completely different:

Summary
Traceroute to 51.15.241.78
 1 gw-b.develooper.com (207.171.7.3) AS7012  0.242  0.166
 2 gi1-9.r01.lax2.phyber.com (207.171.30.13) AS7012  0.644  0.606
 3 te0-1-0-7.r04.lax02.as7012.net (207.171.30.61) AS7012  0.993  3.441
 4 xe-0-1-0-30.r01.lsanca07.us.bb.gin.ntt.net (198.172.90.73) AS2914  0.688  0.676
 5 ae-19.r00.lsanca07.us.bb.gin.ntt.net (129.250.3.235) AS2914  1.090  0.826
 6 ae-0.level3.lsanca07.us.bb.gin.ntt.net (129.250.9.94) AS2914  0.764  0.907
 7  *  *
 8 (212.3.235.202) AS3356  141.116  141.111
 9 (195.154.1.119) AS12876  141.286
 9 (51.158.8.180) AS12876  140.387
10  *  *
11  *  *
12  *  *
13  *  *
14  *  *
15  *  *
16  *  *
17  *  *
18  *  *
19  *  *
20  *  *
21  *  *
22  *  *
23  *  *
24  *  *
25  *  *
26  *  *
27  *  *
28  *  *
29  *  *
30  *  *

Traceroute to the “bad” one, however, looks the same as that to my server.


#66

Hi,

the mysterious thing is that both have ipv6 connectivity and the score is about 18.x
The “good” one lives in DC Paris
The “bad” one lives in DC Amsterdam

ps: i’m the author from the other thread
pps: the bad one is gone. I setup a new one in Paris - maybe this one is better


#67

So it looks like routing problem between Scaleway DC AMS and LA monitor station now.


#68

Maybe just the monitoring station in general. My New Jersey VPS in the USA and my VPS in Hong Kong both dip and dive.

Noah
https://logiplex.net


#69

Scaleway used to have an issue on part of their network that corrupted the checksums on about 50% of outbound UDP packets. No joke.

For the person I heard about it from, Scaleway fixed it around September, but maybe other parts of their network are still affected.


#70

I have done some test to my server from outside Scaleway, mostly NL/HK/US. In most cases there is no problem. Although on some of the test servers there are a few packets lost, it is not like the continued “i/o timeout” the monitor has seen.

An interesting thing is that I did not get any reply on one particular server in US, and traceroute to my NTP server on that machine shows almost the same route as from the monitor.


#71

I am seeing this same behavior on a Digital Ocean server in NYC. It happened a few months back, and has now happened at least twice in the past 2-3 days. I joined the beta server pool and it’s happy as a clam. over a time period that the LA monitoring server was scoring my server around 0. Everything about the hosting network looks good to me, as does my server.


#72

My NYC1 droplet’s IPv4 address’s score went as low as -76 yesterday. (It’s more or less okay now.)

https://www.ntppool.org/scores/159.203.158.197

I was able to confirm that the issue didn’t just affect the monitoring server. (But I didn’t entirely track it down, or report it to DigitalOcean.)

Edit: It has two IPv4 addresses in the beta pool. The non-floating IP maintained a perfect score.

https://web.beta.grundclock.com/scores/159.203.158.197
https://web.beta.grundclock.com/scores/159.203.186.41


#73

My FR dedicated server with online.net (by scaleway) seems to have similar issues. Since it has the same configuration as my hetzner servers (which mostly keeps at score 20), I believe it’s hardly a configuration issue from my side.

https://www.ntppool.org/scores/62.210.205.24

It looks better than the above charts from scaleway VPS though.