My FR dedicated server with online.net (by scaleway) seems to have similar issues. Since it has the same configuration as my hetzner servers (which mostly keeps at score 20), I believe it’s hardly a configuration issue from my side.
https://www.ntppool.org/scores/62.210.205.24
It looks better than the above charts from scaleway VPS though.