I would suggest a TTL of 900~1800, that is more then enough, there is no need to change more often.
In my opinion the goal of the NTP-DNS-system is to take wrong/bad tickers out, but not many clients request all the time.
As you always present 4 IP’s, the chance to get all 4 bad is not great. Sure it can happen.
There is no point to set it much lower unless it’s feeding stupid systems that poll every 10 seconds and abuse the system.
Heck my server ignores systems that poll too much.
Just improve the DNS so it will take out systems that are proven to be unreliable too often that they are less presented.
Put a counter on their ‘mistakes’ and make a stepping-system that take them out or put them back.
Like: Wrong ticker = out instantly => but can return after 10 right ticks.
Unreachable = step down, not taken out, testing 5 more times, nothing => out but will return once 5 reachable tested ok.
With such an approach it doesn’t need to be that low on TTL and you save a lot of DNS-traffic.
Just my opinion.
Bas.