Site to evaluate NTP servers

1

Folks,

My students at TU Delft have developed a website as part of the course which evaluates the accuracy of NTP servers.

We are currently on beta phase – so any feedback is more than welcome

What does it do?

  1. It sends NTP queries from two vantage points:
  • From our gps-synced NTP client in the Netherlands
  • From a RIPE Atlas in your vicinity (it tries to find a probe close to you both geographically and topologically)
  1. It provides a comparison of both, showing NTP metrics and you can download the results

The tool is also open source

Thanks

7 Likes
2

hi giovane,

Nice tool! Thanks for sharing.
Just tried a few servers (including my own) and here are my observations:

  • For some reason all the servers (public as well as my own) appear to be located somewhere in the Atlantic. Burmuda Triangle anyone? :wink:
  • TimeNL-derived precision of my server looks rather high (2E-18). This is unlikely to be the case, since it is not an atomic clock itself, but just syncs to GPS or even DCF/PZF. Expected precision is 10E-9 to 10E-10 at the best with the poor GPS reception. Or is this the precision of your server?
  • Precision from RIPE Atlas probes looks rather low to me (4x10E-6).

Kets

2 Likes
3

The server I tried got tested from time.nl (94.198.159.31) and an Atlas probe on the same network 94.198.159.9, so that was less distributed than I imagine was intended. :slight_smile:

2 Likes
4

Please make it clear which fields are simply reporting the target server’s response contents: precision, stratum, etc. The offset and RTT are values observed at your NTP clients.
I’m unsure how jitter is being measured.

The formats could be more consistent.
precision is reported in both base 2 and base 10
reference ID is reported as dotted decimal and hex.
root delay is reported with different numbers of significant digits
[The values should be reasonably rounded.
A root delay of 0.0043640099465847015 should have no more than 1-2 significant digits]
If multiple formats are desired, then report both for each measurement.

In NTP requests for similar studies I replace the reference ID by a fixed string, e.g., INFO
This enables the NTP server to detect requests from the probes.

2 Likes
5

Results should include the UTC time of the measurement

image
image754×522 45.3 KB

The offset above is reported as 1.07 seconds, which is wrong.
I suspect there is a too much delay in the delay between the Mode 3 request is populated and the actual transmit time.

6

thanks a lot, @Kets_One

For some reason all the servers (public as well as my own) appear to be located somewhere in the Atlantic. Burmuda Triangle anyone? :wink:

Fixed. (for now). Reason: it got rate-limtied on how many times a day it could download geoDB from maxmind, so there’s indeed a Easter egg that the students used for geolocation – also for Anycast servers, which is the Bermula Triangle :slight_smile:

I’ve created an issue on GitHub for that

TimeNL-derived precision of my server looks rather high (2E-18). This is unlikely to be the case, since it is not an atomic clock itself, but just syncs to GPS or even DCF/PZF. Expected precision is 10E-9 to 10E-10 at the best with the poor GPS reception. Or is this the precision of your server?

Thanks I’ve created another issue for that.

Precision from RIPE Atlas probes looks rather low to me (4x10E-6).

These values are automatically reported by Atlas. For instance, you can click on Measurement ID under the Atlas results, and it will open the measurements from Atlas
As an example, see this I just ran :

https://atlas.ripe.net/api/v2/measurements/116380635/results/?format=json

  "version": 4,
    "mode": "server",
    "stratum": 1,
    "poll": 8,
    "precision": 0.0000038147,
    "root-delay": 0,
    "root-dispersion": 0.000152588,

So I’ll have to check with Ripe Atlas – they have a mailing list, I’ve just posted it there:

BTW, there’s an issue with offset computed by Atlas, see the same e-mail thread above

Thanks a lot

1 Like
7

The server I tried got tested from time.nl (94.198.159.31) and an Atlas probe on the same network 94.198.159.9 , so that was less distributed than I imagine was intended.

thanks for testing it, @ask

To which country your IP address was mapped to when you ran the measurement? Was it from the Netherlands?

So the way it works is like this:

  1. It gets your IP address from the browser
  2. It looks it up its geolocation on the maxmind file in disk
  3. It tries to get RIPE Atlas probes in your vicinity, first topological and then geographical

Why? Because the goal is to measure a server’s accuracy closest to your location, so it can be more representative.

If your IP was in the Netherlands, then I think it worked well. If not, then we need to debug this.

But SIDN hosts an Atlas Anchor on the same network as the backend, same geographical location. In your case, pretty much it measured the same thign.

I’ll create a Github issue to NOT use probes on the same backend prefix.
edit: here is Github Issue

But what should matter is your own IP address.

8

thanks very much, @stevesommars

Please make it clear which fields are simply reporting the target server’s response contents: precision, stratum, etc. The offset and RTT are values observed at your NTP clients.
I’m unsure how jitter is being measured.

Thanks, new issue created

In NTP requests for similar studies I replace the reference ID by a fixed string, e.g., INFO
This enables the NTP server to detect requests from the probes.

Oh nice. But for Atlas, we cannot do it, we could in principle do it for our time.nl synced server. But why would you think is important to disclose this piece of information to the NTP server?

I mean, an attacker could easily decide to tune its response based on the client with this information

thanks a lot

9

I believe the precision refers to that of the system clock, not the reference clock. I.e., mostly impacted by the speed of the processor. E.g., ntpd on startup runs a test loop to determine the time it takes to execute, and then derives the precision value from that. And depending on the respective load on a system, the value can differ between startups.

RIPE Atlas currently uses the NULL value (0x0) as reference ID (memory to hold the packet is nulled, and reference ID is not among those fields that are subsequently set before sending). Might be worthwhile raising a PR with them to set the reference ID to some more specific value, e.g., “ATLAS”.

10

Phillip (who used to work for RIPE) responsded:
“Looking at the code, the precision field is just copied from the reply that
was received from the server. Unless there is a decoding error.”

11

4x10E-6 is 2^-18. Not great, but also not outlandish. Probably a smaller device.

E.g., ntpd on my RPi 3 assesses that to be the system clock precision. chronyd is a bit more optimistic, assessing the system clock precision to be 2^-23 on the same type of hardware (though not usually this much of a difference, i.e., CPU frequency scaling in effect when the tests are run and other aspects also play a large role).

12

It was a globally distributed service that probably does have servers in the Netherlands, so your diagnosis makes sense.

1 Like
13

I have tested one of my ntp servers, tick.sanren.ac.za (155.232.19.6, 2001:4200:7000::6) on both IPv4 and IPv6 from my desktop (146.64.84.9, 2001:4200:7000:4::9). All the addresses are in South Africa. Also according to Maxmind. The Atlas Probe is from Netherland though, (185.21.188.122 and 2a0a:6044:b304::2a).

14

I have tested one of my ntp servers, tick.sanren.ac.za (155.232.19.6, 2001:4200:7000::6) on both IPv4 and IPv6 from my desktop (146.64.84.9, 2001:4200:7000:4::9). All the addresses are in South Africa. Also according to Maxmind. The Atlas Probe is from Netherland though, (185.21.188.122 and 2a0a:6044:b304::2a).

Hum, that’s odd.

So I tried to reproduce it by configuring a Tor browser to use a ZA exit node.

  • IPv4: 102.211.56.12
  • IPv6: 2c0f:6c0:0:2::6983

I had no issues with geolcation. See screenshot

tor
tor896×769 61.2 KB

Weirdly enough, if I query these IP aboves on Maxmind’s site, it nows say it’s from Egypt:

15

The round-trip time of the Atlas probe in your screenshot is 176ms. That is pretty far way.

Here is what I see:

image
image878×713 75.6 KB

image
image864×706 71.4 KB

16

Wow, spot on! thanks so much @john1 for spotting this bug.
TL;DR: It’ s been fixed. It always chose probes from NL, now it chooses from the client side

Here’s what happened: you are right: all the atlas probes were from Holland, you can see them at:

image
image812×286 25.2 KB

So the students fixed it for us (thanks tons @Serban) and I pushed it today.

I just tried again with your server , and got all probes from ZA

image
image828×669 53.2 KB

See the RTT?

Now check the probes on atlas’s own site:

image
image1084×636 40.6 KB

The URL for it is: RIPE Atlas - RIPE Network Coordination Centre

thnanks a lot once again @john1

2 Likes
18

there was a bug, see Site to evaluate NTP servers - #16 by giovane

so maybe if you try again you’ll get better results

19

Thank you @giovane, it looks good now. I checked both IPv4 and IPv6. The IPv6 test looks better than the IPv4 one for all my servers. It looks like the IPv6 probe is at a peering point, while the IPv4 probe is deeper in another network.

image
image874×492 56.9 KB

image
image817×492 52.8 KB

1 Like
20

thanks @john1
Bear in mind, however, that Atlas currently has a bug with offsets , so it inverts its sign

(we will update our site to not use the offset aclculated by atlas)

1 Like
21

FIXED on our website

1 Like