I think I’ve found a bug in the scoring visible only in the unusual case of a monitor and a monitored server being behind the same misbehaving firewall.
For some reason, this system is unreachable from the internet despite being in the same /64 as others which are, affecting both ICMP echo and NTP. It is also listed in the test monitoring system, where all monitors save its sister can’t raise a response and so score it near -100:
The CSV log shows the wierdness. It appears the other systems listed under “active” are in fact polling much less frequently, as one would expect for monitors listed under “testing”. More importantly, the overwhelming near-consensus loses out to the one monitor which has never had a problem getting a response in the overall score (recentmedian) calculation.
ts_epoch,ts,offset,step,score,monitor_id,monitor_name,leap,error
1716545449,2024-05-24 10:10:49,,1,20,64,recentmedian,,
1716545449,2024-05-24 10:10:49,-0.000040909,1,20,87,usiad1-2y4swg5,,
1716545262,2024-05-24 10:07:42,,1,20,64,recentmedian,,
1716545262,2024-05-24 10:07:42,0,-5,-91.691902161,83,sgsin4-egctcm,,network: i/o timeout
1716545138,2024-05-24 10:05:38,0.000159017,1,20,87,usiad1-2y4swg5,,
1716544781,2024-05-24 09:59:41,,1,20,64,recentmedian,,
1716544781,2024-05-24 09:59:41,0.000176351,1,20,87,usiad1-2y4swg5,,
1716544746,2024-05-24 09:59:06,,1,20,64,recentmedian,,
1716544746,2024-05-24 09:59:06,0,-5,-91.949012756,65,sgsin1-egctcm,,network: i/o timeout
1716544405,2024-05-24 09:53:25,,1,20,64,recentmedian,,
1716544405,2024-05-24 09:53:25,0.000022319,1,20,87,usiad1-2y4swg5,,
1716544204,2024-05-24 09:50:04,,1,20,64,recentmedian,,
1716544204,2024-05-24 09:50:04,0,-5,-91.666862488,52,ausyd2-1spwqxm,,network: i/o timeout
1716544070,2024-05-24 09:47:50,0.000005368,1,20,87,usiad1-2y4swg5,,
1716543761,2024-05-24 09:42:41,-0.000020407,1,20,87,usiad1-2y4swg5,,
1716543447,2024-05-24 09:37:27,,-5,-95.936134338,64,recentmedian,,network: i/o timeout
1716543447,2024-05-24 09:37:27,0.000225635,1,20,87,usiad1-2y4swg5,,
1716543179,2024-05-24 09:32:59,,-5,-95.936134338,64,recentmedian,,network: i/o timeout
1716543179,2024-05-24 09:32:59,0,-5,-91.666862488,81,kricn2-egctcm,,network: i/o timeout
1716543134,2024-05-24 09:32:14,,-5,-95.936134338,64,recentmedian,,network: i/o timeout
1716543134,2024-05-24 09:32:14,-0.000014508,1,20,87,usiad1-2y4swg5,,Notice it’s not consistent: there are recentmedian entries nearer -100 toward the end of the above .csv snippet.