Monitors belgg1-19sfa9p and belgg2-19sfa9p having hiccups?

I would really appreciate fellow time-nuts who are more network-savvy to help out the less savvy ones (like me) with setting up a professional router.
Maybe an idea for a new thread?

Correction to that, I wasn’t thinking this through. It actually depends on what exactly you need whether this would help, or not. Apart from not knowing how AVM implemented the functionality.

When talking about NAT in this context, it’s mostly DNAT, or port forwarding. E.g., making the NTP port on an internal device reachable from the outside. With IPv4, that can only happen for a single device, anyhow (when you only have a single external, aka “public” IP address). So there should not be an issue to have multiple devices that only have outbound communication needs. Or internal devices that expose different services (aka, transport protocol ports) to the outside world. Say, one NTP server, one webserver, …

So if you want to expose only one NTP server, that should work with the exposed host feature. If it is multiple NTP servers, but only one public IP address, then it wouldn’t. But then I am not sure (or missing information) how you do it today.

Think you described it well. Currently i run three IPv6-only NTP servers, so no issue with the one IPv4 address dictating only one possible IPv4 exposed host.
Looking to add NTS to one of them.

Not sure a “professional” router would be needed. The requirements, and context, are not clear to me at least, e.g., the question as to how many internal NTP servers on IPv4 should be served, and how many external/public IPv4 addresses are available.

If it is a matter of one IPv4 address only, and thus one NTP server only as well, the exposed host concept of the Fritz Box might be the (a) way to go. Depending on how much traffic you want to absorb. And unfortunately at the cost of having to handle the security side on the exposed device yourself.

Another, somewhat easy to manage approach (in my book, YMMV), might be to play with OpenWrt as router. It runs on not too expensive devices, it has (in my view) a somewhat comfortable interface for many common things, but has the “power” (in the sense of functionality, and more importantly, user access to that functionality) of an almost full Linux system, e.g., the stateless NAT that @NTPman is referring to.

Or combine the two: Keep the Fritz Box as router, define an exposed host, and have that exposed host run OpenWrt, which has a decent set of security defaults, owing to it being intended to act as router, i.e., be exposed to the Internet, and protect internal devices. The OpenWrt box could run NTP itself, with limited options though to have external hardware feed time into it, like GNSS receivers. Or use it just as another box in front of the actual NTP server, to handle security. But more devices obviously make things more complex.

But also setting up a decent firewall on a generic Linux system is doable.

So it all really depends on what you’re looking for, how comfortable you are with those various aspects, etc.

Mikrotik :+1:

3 Likes

Yeah, that manufacturer has been in the back of my mind as well throughout this discussion. Powerful (in the sense of functionality for money) and affordable devices, am using a few of those myself. Would certainly qualify as “professional” at least from the feature set point of view.

I didn’t mention them because I am not sure as to how they fit the “ease of use” aspect. Though I have to admit I did not usually use them for one of those use cases for which they have those “templates”. That might indeed be a good way to start, and then refine from there, e.g., making the port forwarding stateless. And some aspects are similar enough to Linux (which the devices, like many others, are based on - at least the ones I have used) to be able to leverage available knowledge in that area on those devices, e.g., the firewalling and NAT concepts.

And I have no experience as to their performance capabilities, vs. what people in this thread might be looking for (which I really don’t know, either). From a hardware platform point of view, I think the more affordable ones are similar to what AVM and other vendors use as well. At least the smaller models I am familiar with. But what always stuck with me, and I am not sure whether that is applicable to all their models, is that I had the impression that they are so affordable, and capable functionality-wise, because much of what they do is done in software, on the CPU. And then potentially hit similar bottlenecks as an AVM device would hit as well. But they are certainly professional in the sense that they publish the relevant performance figures for their devices.

But it could be worth a try, similar to the OpenWrt topic. As said, I think the first step would be to see how far one gets when the topic of statefulness is out of the way. I.e., the stateless NAT topic that @NTPman mentioned. Trying exposed host with the Fritz Box might be one way to give that a try, an OpenWrt or MikroTik device, or a Linux box (with or without Intel CPU) other options from which to choose according to personal preferences, access, existing known-how, etc.

Would also be interesting to hear whether anyone is already doing stateless NAT on a device, and how that works out. E.g., when people have multiple public IP addresses that they can assign to their actual end devices (or even just one IP address, but also just one device to connect), and the “router” doesn’t need to do NAT anymore at all, just firewalling, stateless for services exposed to the outside world with potentially a lot of traffic, such in case of an NTP server. And stateful firewalling for the occasional outbound traffic, like upstream NTP server contacts, web browsing, software update downloads… Maybe even running a monitor, but I never got a response to my question above, so not sure what demands a monitor places on the router.

do ARM processors have interrupt controllers? - Google Search has contradictory information. Apparently the ARM processors have a single IRQ line, but a given ARM-based system may or may not have additional interrupt controller to allow more than one device to interrupt. That interrupt controller is a standard part of Intel/AMD chipsets used with the processors.

There’s not much they could do without breaking most other UDP servers behind NAT. They could micro-optimize for NTP by marking the mapping to go away after it’s used once, but at the cost of slightly slowing processing of other UDP by the comparison to NTP’s port 123 and the check for that marking on the return path.

I’ve had good results with the UniFi Dream Machine Pro. It’s $400 here in the US, but it’s also useful for managing multiple UniFi access points centrally instead of needing to configure each AP individually. It also can centrally manage UniFi cameras, both doorbell and security cameras, and some other systems like card reader/door locks. It provides VPN server functionality for WireGuard, OpenVPN, and standard VPN protocols.

Tech support is via https://community.ui.com/ where much of it is peer-to-peer, but there is some participation by Ubiquity staff.

Of course, IPv6 NTP service doesn’t have the NAT overhead. I am still devilishly curious why @Ask has resisted turning on IPv6 responses for all zones, not just 2.*

2 Likes

Should’t that rather be “doesn’t need the NAT overhead”?

If connection tracking as such is enabled, e.g., because of a baseline stateful firewall, on Linux, just adding a rule to accept incoming NTP clients doesn 't disable that, even if it is not strictly needed. It takes the extra step of exempting those packets from connection tracking, requiring two extra rules.

A vendor of an “integrated” device (where the user can just configure pinholing, but not how that is implemented “under the hood”, like is my suspicion is the case for those AVM boxes) might just not set those two extra rules, maybe on purpose, maybe not knowing or caring. Users who use IPv6 pinholing for most services other than exposing NTP servers for the pool will probably not even notice the difference from a functional point of view.

The original claim I was referring to was that “ARM-type CPU’s have no hardware interrupt-lines and have to poll the busses/chips” (emphasis added by me). May seem like nit-picking, but seeing what further claims were based on that one, I found it relevant to clarify.

1 Like

Looked at that one, too expensive for me. Sadly.

The pool monitor places very light load on both the computer and the router. Even with both IPv4 and IPv6 monitors running against the test system (grundclock) and the production pool, it amounts to no more than a handful of NTP query packets and communication with the monitoring infrastructure each second. It can easily be tacked onto a server with other duties, the most important requirement is low jitter connectivity, in other words, where the delay communicating with any NTP server almost never varies due to local or ISP bottlenecks on the path.

1 Like

Quite so. I should have added that x86 processors also have only a single IRQ line at the processor, the difference that can lead to confusion such as @Bas apparently suffered is the interrupt controller which allows multiple devices to share that CPU interrupt input line is essentially always available as part of the associated chipset, while ARM-based systems sometimes have no interrupt controller as the system designer may be satisfied with software polling to keep hardware cost and complexity lower, or have only one device that needs a hardware interrupt.

1 Like

There are affordable alternatives. If you have an old unused PC from the last 20 years or so, or buy one, Linux (or OpenWRT) will give you a very powerful router. You might have to add Gigabit Ethernet or two, but they’re inexpensive. With x86 OpenWRT, you get the plug-and-play web UI to make it easy to configure like the FritzBox, and I bet the installation is pretty easy as well, though I haven’t done it. Any old PC with 4GB RAM, 1GHz or better CPU, and any sort of disk or SSD will work beautifully.

In the US at least, it’s easy to find powerful, compact used PCs pulled out of service because they’re relatively old but still quite capable. If you look specifically at Point-of-Sale (POS) devices they often already have two GigE interfaces. For example, I searched for “POS PC” on eBay and then sorted lowest price first, and the first result in the list is:

Full specs for that system can be found at Qotom-Q305p Mini PC with Intel Celeron 3205u Processor Dual Core 1.5 GHz, Linux Fanless Mini Desktop PC - 2 Ethernet Mini PC and Barebone Mini PC price

With two gigabit ports, 4GB RAM and 32GB SSD, and a price including shipping in the US of $67 it’d make a fine OpenWRT box for little money and a lot more CPU than the traditional consumer-router systems OpenWRT is known for.

2 Likes

Even MikroTik’s “professional” RouterOS is available for Intel/AMD, though not for free. But it can easily be tried for free for 24 hours.

I should have mentioned the Qotom also already has the WiFi which I assume the FritzBox is doing for you. If you go via the old PC route, you’d probably need to buy a WiFi adapter compatible with Linux. If you do, and you want the best WiFi, look for at least 2x2 MIMO (preferably 4x4) and long antennas that won’t be blocked by the PC itself. For 4x4, that would mean 4 antennas.

I forgot to mention it because I think of NAT/firewalling and WiFi as two distinct functions, though for many people “router” implies WiFi and NAT. For decades I’ve shopped for them separately, buying WiFi Access Points (APs) separately from routing/firewalling equipment, sometimes configuring a router’s built-in WiFi to essentially disable it (garbage or hidden network name, complicated password I don’t record).

Thanks Dave,

I keep it in mind, but now DSL and Fritzbox are splitted and it has not had any troubles.
Everything is fast and the monitor (IPv4) works well, handing out good values on severs that I checked.
Where before it was all over the place in dots.

Tommorow I will enable IPv6 again, and see what happens.

Something is wrong in the Fritzbox when it’s doing DSL-modem functions as well.

Since it’s closed code, not much to check.

Fingers crossed :ok_hand:

I was traveling and didn’t see this until now, but wanted to add that the scoring system (with help from @stevesommars & @mlichvar) was designed to deal gracefully with some monitors having hiccups.

I’ve been working on an update to the monitor to support more monitoring data, better logging/diagnostics and easier management of them.

Currently all monitors are testing all servers occasionally (“testing”), and for each server the system tries to choose the best suited monitors to do more frequent tests (“active”). It might make sense to add a third category of “standby” monitors that just test a few times a day (or not at all) and then choose the “testing” monitors from that.

The code to do the selection is here: monitor/scorer/cmd/selector.go at main ¡ ntppool/monitor ¡ GitHub

The scorer more generally is here:

1 Like

Just for clarity, the “issue” here was not a functional one of the system not dealing gracefully with monitors having hiccups. With hiccups in that context probably meaning, e.g., not providing data.

This was only about the offset and score graphs being blown out of proportion to be neigh unusable. I.e., the monitor seemed to be working properly, but because of networking issues on the monitor side, the actual measurements had issues. Namely very occasional offsets of up to one second, or more, causing the aforementioned distortion in the graphs.

Just a single value that is off during the time window of the graph can cause this. Which is “ok” when it is an actual issue with the monitored server. In this case, it was known that the monitor side was the one having the issue, which is a bit more irritating then when that causes the graphs to become essentially useless (at least I make heavy use of the graphs to quickly see whether one of the servers needs attention).

Regardless, one thought that was raised in this thread was to make it a user option to be able to somehow scale the graphs, e.g., have some control element that allows to ignore rare, but strong outliers to be able to focus more relevant parts of the graph, but not ignore them entirely.