Disabling the clientlog can decrease the CPU load and improve the maximum packet rate slightly (maybe 1%).
Increasing the limit can be useful to better support the interleaved mode. If the limit is too small, a client polling at 1024s interval might not be getting interleaved responses as the previous transmit timestamp was already dropped to make room for a different address. From what I have seen, very few clients have the interleaved mode enabled. Starting with chrony-4.2, the interleaved-specific data is separate from the addresses and it can work on a busy server even if the limit is small.
On my servers I have the limit set to 100000000. There is a script which looks for abusive clients every 30 minutes. If one address sends over 500k packets in that interval, a 10s tcpdump capture is made so I can investigate it and it doesn’t consume much of the disk space.