So some population of IPv6 client(s) is flooding your server. In your position, I’d want to know more about how many different addresses, and see if they are also serving NTP, or, more likely, it’s some broken SNTP client you might be able to get fixed by the operator with help from their ISP and/or the responsible software developer. If you share some offending IP addresses here, others can see if they’re getting hit as well.
You can use packet capture, or if you happen to use ntpd/ntpsec, ntpq -ncmru | head -50 after disabling the firewall rule.
Where do you get these statistics from? My vantage point (DNS-level information) paints an entirely different picture (although Belgium is hard for me to research), with usually only about 5% of NTP servers operating IPv6. I suspect Belgium will be higher though.
Your personal traffic measurements are encouraging, though.
I get them from my MikroTik router, which reports packets/s and also shows a graph. I’ve seen combined IPv4/6 peaks of up to 2200 packets/s. My IPv4 speed is set to 1 Gbps relative to others but a few days ago I changed the IPv6 speed to 2 Gbps (from 1 Gbps) relative to others.
In this pool, Belgium currently has 22 IPv4 servers and 18 IPv6-enabled ones.