Bas, your problem isn’t the bandwidth. Your problem is that you are doing NAT on your router. You have been here long enough so you should know that NAT may cause issues.
The point of my previous “chronyc serverstats; sleep 1h; chronyc serverstats” command was to get two measures of sent packets so that the delta could have been divided by 3600 to get the average queries per second over one hour. A shorter interval could have worked as well. But this point is now a little bit moot now that your server has been set to “monitoring only”. The average rate would have given us an exact value of how much queries you are actually getting. My worst case guesstimate would be around 130 queries per second when a Belgian server has been configured with a 512 kbit/s netspeed setting. 130 qps is honestly not that much bandwidth-wise. NTP packets are only around 100 bytes.
I believe your Internet connection is VDSL so you will need some sort of a modem, like the one you have now. However, if you configure it to “bridge mode”, disable its firewall and NAT, and assign the static IP address you have to your NTP server instead of the router, it is likely that your setup would handle much higher query rates. You would probably still need to assign some 192.168.x.x address for the router for management purposes.
If you have multiple IP addresses available (like a /30 subnet), you can assign one IP address to your NTP server, and the other IP address to some other router that does NAT, both behind the same bridging modem. This other router (like a regular wifi router without VDSL features) could handle all the other devices in your home network. This is only one example, there are other ways to achieve similar results. You will need to take care of firewall issues one way or another, though.