As an additional data point, I started pinging the server too. Although these results are still preliminary, it looks like NTP packets get dropped more often than ICMP echo requests (pings).
Ping stats: Singapore, Netherlands, Australia, Finland.
Edit: There’s also a summary stats page.
You said you had disabled connection tracking, but please verify with “conntrack -L” or “grep port=123 /proc/net/nf_conntrack” (or whatever works for you), and/or “iptables -L -t raw” which should probably show something like:
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
CT udp -- anywhere anywhere udp dpt:ntp CT notrack
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
CT udp -- anywhere anywhere udp spt:ntp CT notrack