I’ve been experimenting with higher traffic servers running on VM’s. I’ve tried FreeBSD and Ubuntu, I’ve played with ntpsec, legacy ntp and chrony. In all cases I max out around 60% cpu and 10-12k qps and then start dropping packets. All the ntp implementations seem to be single thread so adding cores doesn’t help (I tried it just in case the net stack might end up on a different thread, it didn’t). Digital ocean seems to behave a bit better than linode (plus they don’t actually measure the bandwidth) but neither do that well.
If the load were relatively stable I could tune the bandwidth settings but it’s very spiky, I saw a 40k qps 10 min spike on my Bangalore server yesterday. My singapore server has +/- 80% traffic swings in an hour.
Has anybody managed to get beyond the 10-12 k qps range on a VM and if so what setup are you using?
Has anybody done any work on a multithreaded server?