Add more monitoring stations

Thank you for asking. Arnold Schekkerman had a good summary of my concerns and questions about implementing something like this in a recent email thread: http://lists.ntp.org/pipermail/pool/2016-October/007965.html

Typically people ask for this because the California system can’t talk to their server and it seems unfair. However, the point of the system is to take out servers that are potentially possible bad for some clients somewhere, so adding more monitoring systems would likely exclude more servers more of the time.

That being said, it’s something I want to do, but there’s some outstanding work to do first and some decisions to make. I also want the monitoring checks to be at a higher frequency.

Work:

  • Write a better “scheduler” to allow multiple monitors to “eat” at the queue.
  • Rewrite how the monitor data is stored (short term and in particular long term). Currently they’re just in an SQL table and then moved to an archive table after X weeks. The archive table is a MySQL table with the “archive” engine and it’s not very reliable (or useful). Adding more data to this system won’t work (not with the current hardware anyway).

Decisions:

  • What happens when monitoring stations disagree? I’d likely go with the one signaling a worse outcome. Maybe with exceptions for countries/regions that aren’t well connected to the rest of the internet (Cough .cn cough)?

There’s also the trouble of just managing more monitoring stations. Either it has to be done very carefully, or code has to be written to decide from moment to moment if a particular monitoring station is to be trusted. The latter could also enable using RIPE Atlas probes for tests.

There are lots of possible implementations of this.

In the email thread referenced above @mlichvar suggested that NTP servers could monitor each other uploading the data. That’d be a whole new type of work and exploration. I recall maybe very early on the system worked like this; or Adrian was talking about having it work like that? http://lists.ntp.org/pipermail/pool/2016-October/007964.html — Anyway, it doesn’t feel like a practical approach to me, but I reference it just to point out that there are many paths to take here. There’s no “just add more monitoring stations” solution that’ll make anything obviously better.

2 Likes