I’ve been experimenting with higher traffic servers running on VM’s. I’ve tried FreeBSD and Ubuntu, I’ve played with ntpsec, legacy ntp and chrony. In all cases I max out around 60% cpu and 10-12k qps and then start dropping packets. All the ntp implementations seem to be single thread so adding…

Quick update. My Bangalore server (3 core 1GB Digital Ocean for $15/month) is now handing peaks of 80k QPS set to gigabit on IP4 and IP6. The @mlichvar rsntp server works very well at these loads. Traffic is around 12 TB/month. [image]

Using rsntp, has anybody seen EINVAL/os error 22 errors? I get anywhere from 4-12 / hour, eg: Jun 14 21:32:11 minime rsntp[6490]: Thread #2 failed to send packet: Invalid argument (os error 22) Jun 14 21:32:31 minime rsntp[6490]: Thread #2 failed to send packet: Invalid argument (os error 22) Ju…

@mlichvar cc the post above: [image] Getting beyond 10k qps? Server operators Using rsntp, has anybody seen EINVAL/os error 22 errors? I get anywhere from 4-12 / hour, eg: Jun 14 21:32:11 minime rsntp[6490]: Thread #2 failed to send packet: Invalid argument (os error 22…

I get that too and I think your guess is correct. It’s probably trying to respond to an invalid IP address. If you pull from git, the error message will include the remote address.

[image] mlichvar: I get that too and I think your guess is correct. It’s probably trying to respond to an invalid IP address. If you pull from git, the error message will include the remote address. Awesome, thanks! That’ll definitely help debug the errors!

[image] avij: My CentOS 7 server in Singapore handles an avg of 30k qps per day, with 5 minute peaks of 50+k qps with standard ntpd How do you change port in ntpd so it listens on port 11123 that rsntp reads from?

I don’t think you can change the port in the standard ntpd distribution…

I don’t think you can change the port on ‘ntpd’, but you CAN use the -I option when starting ntpd to make it bind only to 127.0.0.1 (on 123), Then when you start rsntpd, you do something like: -a 192.168.1.5:123 -s 127.0.0.1:123 Change the IP in -a, obviously :slightly_smiling_face:

Thanks, I got it to work but I have to use one more public IP that ntpd itself can use. In ntp.conf I changed from interface listen mypublicip-1 to interface listen 127.0.0.1 interface listen mypublicip-2 interface ignore wildcard and then I start rsntp with rsntp -4 4 -a mypublicip-1:123 -s…

Q1: The IPv6 address specified by the -b option needs to be in brackets, e.g. [2a03:8600::dd]:123 Q2: Something is probably listening on the IPv6 port 123. ss -u -6 -l -p should print what it is. Q3: Maybe it is still too much for the CPU? If it’s fully loaded, the receive queue might be full and…

Getting beyond 10k qps?

Server operators

littlejason99 October 13, 2018, 10:10am 28

… and yet their “Contact Us” page is broken. Guess they don’t need any more business?

Adding servers to the China zone

Topic		Replies	Views
Oddities with rsntp on CentOS 7 Server operators	10	1086	April 22, 2019
The issue of NTP requests exceeding bandwidth load Server operators	54	1578	November 24, 2024
Some client really can't behave Server operators	54	2382	December 18, 2023
Network: i/o timeout Server operators monitoring	32	894	November 20, 2024
I take a big hit in queries Server operators	111	1604	December 15, 2025

Getting beyond 10k qps?

Related topics