Now that the server’s score dropped below 10 and it’s no longer in the pool, the server seems to respond to pings just fine. This does not point to problems with international connections, but it’s probably something closer to the server itself. Maybe it’s the server, maybe it’s the ISP.
Let’s check the basics first. Do you see anything in “dmesg” output about dropped packets? Or /var/log/messages? Does “conntrack -L” show a long list of IP addresses sending packets to port 123? See also this post about connection tracking.
Besides that, it looks like 47.92.198.198’s netspeed setting could be set lower. I’m not sure what the netspeed has been set to at the moment, but looks like it is currently at least 12 Mbit/s. Try the lowest 512 Kbit/s setting first and observe the results.