Help! Pool Server DDoS! Has my IP got Hardcoded?!

Server operators
1

Hello, I’m the operator of Pool Server 106.187.50.120. This server was set to a “Netspeed” of “20 Mbps”, previously it received little traffic, and recently joined the effort to improve the NTP availability in China. Since join, there’s a slight increase of traffic, the actually traffic was consistently around 4 Mbps and running without issues for months.

Around 2017-02-23 12:36 UTC, a great increase of traffic (70 Mbps!) is observed.

829×397 28 KB

and my provider, Linode, blackholed my IP address at once. They told me the traffic was like this:

It looks to be an NTP flood. Here is a sample tcpdump for your reference:

07:36:02.199255 IP 101.17.121.231.55412 > 106.187.50.120.123: NTPv3, Client, length 48
07:36:02.199467 IP 106.187.50.120.123 > 101.17.121.231.55412: NTPv3, Server, length 48
07:36:02.199778 IP 123.122.71.216.47191 > 106.187.50.120.123: NTPv3, Client, length 48
07:36:02.200328 IP 121.235.18.239.38556 > 106.187.50.120.123: NTPv4, Client, length 48
07:36:02.200385 IP 106.187.50.120.123 > 123.122.71.216.47191: NTPv3, Server, length 48
07:36:02.200676 IP 106.187.50.120.123 > 121.235.18.239.38556: NTPv4, Server, length 48
07:36:02.200703 IP 52.69.160.127.123 > 106.187.50.120.123: NTPv4, Client, length 48
07:36:02.201126 IP 106.187.50.120.123 > 52.69.160.127.123: NTPv4, Server, length 48
07:36:02.201925 IP 1.199.74.35.8986 > 106.187.50.120.123: NTPv4, Client, length 48
07:36:02.202051 IP 106.187.50.120.123 > 1.199.74.35.8986: NTPv4, Server, length 48
07:36:02.203584 IP 1.82.184.29.46122 > 106.187.50.120.123: NTPv4, Client, length 48
07:36:02.203816 IP 106.187.50.120.123 > 1.82.184.29.46122: NTPv4, Server, length 48
07:36:02.204172 IP 106.45.215.129.38066 > 106.187.50.120.123: NTPv3, Client, length 48
07:36:02.204476 IP 106.187.50.120.123 > 106.45.215.129.38066: NTPv3, Server, length 48
07:36:02.204644 IP 113.200.107.217.10163 > 106.187.50.120.123: NTPv3, Client, length 48
07:36:02.204818 IP 106.187.50.120.123 > 113.200.107.217.10163: NTPv3, Server, length 48

and

The NTP traffic doesn’t look spurious, that is it doesn’t look like an amplification attack, but it came in an extraordinarily high volume.

So it looks like ordinary clients all cross China?! Look those outdated NTPv3 packets, is it possible to indicate my Pool Server is hardcoded by some ISPs, or embedded manufacturers? I’m so afraid of that…

Or it is just a trivial abuse of NTP Pool servers?

What should I do?

2

I think Windows use version 3.

https://technet.microsoft.com/en-us/windows-server-docs/identity/ad-ds/get-started/windows-time-service/how-the-windows-time-service-works

3

On average, 22% of my NTP pool traffic is NTPv3 with almost all fields set to 0 (the exceptions being version and mode). This traffic matches the fingerprint that the Android NTP client uses. This could be legit traffic.

4

How can it explain the sudden increase of the traffic? Is it possible that an Android ROM hardcoded my IP address…

5

It doesn’t explain the sudden increase of traffic. I assume you’re still blackholed. Is Linode still seeing large amounts of NTP traffic to your IP? SNTP clients against the pool would have moved on long ago.

6

Yes, there’s still a large amount of traffic hitting my IP address, and my IP is still blackholed. I have thought many possibilities for this incident, but it’s all ended up with the guess of IP hardcoding, it cannot explain the sudden spike anyway.

7

Could it be an large ISP’s DNS server that does not respect the TTL on DNS records?

8

Yes, it could be result of caching from an upstream DNS server in China… But the traffic should increase gradually to the maximum, not a single spike (it is still possible if there’s any NTP redirect/hijack within an ISP).

And I have checked some popular DNS service with no success.

9

Do you still see the traffic or did it go away?

10

Yes, the traffic is still here.