I receive quite some, more than 200/sec, NTPv4 requestes on port 123 TCP, not UDP.
I am not sure what to make of this, as it happens on all servers I operate.
Another, odd thing is the quite often seen ICMP echo/reply messages. It seems many people “ping” the pool for unknown reasons. Just curious though if you have the same experience with your servers.
This is pretty normal for any host in the pool, although not necessarily at that volume. I just took a 1 MB sample of traffic, and for 8225 UDP port 123 packets, I received 5 TCP port 123 packets. It’s not NTP, but could be misconfigured daytime protocol, or just random port scans.
Pings to a pool host is pretty normal too. I can’t see any harm in allowing them. I get far more “port unreachable” blowback for replies to (possibly-spoofed?) requests than I do ping requests.
The incident rate of TCP 123 to other ports is about 5000 to 1, so portscans are most likely not the root cause for packets to TCP 123, even if they are a minor part the TCP 123 packets (about 1/5000)
I investigated this a bit more and found that quite some of them are in fact ntpv4 client requests which are just not UDP but TCP for some unknown reason.
It looks like some router/firewall is changing the UDP packet into a TCP one, while some of them are sending a TCP syn of 0 length, others send a TCP with full ntpv4 data body.
The 0 length TCP syn packets get re-sent usually 2 or 3 times about 1 second apart from each other.
I wonder what will come out of it if I answer them syn-ack, instead of just dropping them…
How can it be NTP? RFC5905 clearly specifies that NTP is UDP. Are there actually implementations out there which are so broken that they construct TCP queries and expect replies?
Yes, it seems there are clients that send valid NTPv3 requests over TCP and they receive data from the server. I have no idea why they do that, but I’d be interested to know if it’s a bug or feature.
There are two types of “TCP NTP clients” out there.
#1 regular UDP NTP clients, where a company firewall-thingy translates UDP traffic to TCP for some unknown reason, but there are systems to do exactly that on the market.
#2 Some awfully misconfigured routers which send TCP syn packets on port 123 and try a handshake waiting for a syn ack response.
I will evaluate this further since the second case seems to be more regional. I spotted Vodafone Portugal and Vodafone Germany IoT connections doing this. May be a home device / IoT / router bug which causes case #2
Can you give any examples of products which do #1 or references to seeing this traffic in the wild? I’m less curious about #2 because it seems likely to be a bug born of ignorance, but it would be good to know if you find out which makes & models do this.
If there is no listener and the TCP SYN carries no data, we can’t determine what the intended connection is.
If there is no listener and the TCP SYN carries data, we can make an educated guess about the intended connection. Aside: packet analysis tools like Wireshark can use heuristics and may guess incorrectly.
If there is a listener, the TCP connection payload can be analyzed. Wireshark must be told that this is TLS to do a valid dissection, if NTS is the payload.
Plus one needs to distinguish the server and client ports. I sometimes see UDP probes with destination port=Netbios & source port = NTP. I suspect these are attempts to defeat firewalls.
There will be a listener. I will do a NTS-NTP server setup tomorrow, which will listen to both TCP 123 and TCP 4460 and capture the traffic to see if it´s indeed NTS traffic or awkward bent UDP ntp traffic.