Actually, that’s not all that surprising that you are having no issues on IPv6. There’s actually no guarantee anywhere that IPv6 and IPv4 take the same route between two arbitrary nodes on the internet. Realistically, the first hop from your server to it’s configured gateway will be the same for both (though this is by convention, not because it’s required), but beyond that, it’s impossible to say (and depending on where the two nodes are located, it may be almost guaranteed that they take different routes).
I regularly see cases with my servers where only IPv4 or only IPv6 shows a few packets lost.