Joining the pool kills my Internet

Yes, perhaps. The implied Network Address Translation (NAT) is from the router’s external, WAN-side IP to an internal LAN-side IP. I’m not modifying that from default.

What I am doing is specifying some ports to be forwarded and all the others blocked. That dialogue box is certainly with in the NAT/QOS tab in DD-WRT’s web interface.

It sounds like there’s not a clear solution here.

The connection tracking seems to be an integral part of NAT in it’s various subtleties, at least within DD-WRT. Connection tracking is also limiting max NTP requests per second.