Provided you have the ability to unilaterally change those hostnames without requiring any action from your customers (i.e. you aren’t relying on them changing a setting, accepting a firmware update etc) then that sounds fine . Basically, if you accidentally ship something like this, or this, you need to be able to get it the heck away from pool infrastructure and servers reasonably promptly (e.g. by temporarily diverting it to your own servers, or to another time source, or turning NTP off entirely, until the issue is rectified).
A caveat to watch out for - HTTPS isn’t a reliable provisioning method for devices that may have an incorrect clock at boot. Whatever you use needs to work regardless of what it thinks the time might be, which rules out most of the usual X509 certificate-based tools, including anything involving TLS.