Random querying suggested that each of the four servers were regularly being published in DNS for the cn and asia zones, even during periods of little activity.
So, I did a bit more digging with the result that the bursts in traffic are coming in from a few operators in India, rather than China (details are below after the break). I guess this is a combination of higher IPv6 in India, a dire lack of NTP servers in the India zone, and my servers being in the asia zone.
An outstanding question though is why the traffic arrives in such harsh bursts across different servers at the same time:
All four of the servers get hit at the same time and same period, yet I have other servers in the asia zone that don’t see this pattern at all. If this were DNS related, I might expect to see peaks at different times for each server in relation to its appearance in DNS queries. I’d perhaps also expect to see peaks across other servers in the asia zone.
DNS caching is plausible, but with 4 different operators involved I would expect them to have different DNS infrastructure.
I started with logging IP addresses for about 50 minutes this morning. This included a 20 minute peak period with approximately 2mb/sec of traffic. Overall, I had 790k requests with 452k unique IP addresses. 7k of these were invalid addresses (e.g. they lacked a network part ::1:2:3:4)
After stripping away the interface IDs, I used a script to look up the relevant network prefix and country code through whois. This resulted in another 2k addresses where I couldn’t immediately identify the prefix or code.
Finally, I sorted the results by country code looking at the overall figures and some 5 minute blocks before and during the peak. The top few results are below:
5 minutes of traffic before the peak
- 3678 VN
- 1011 TH
- 786 CA
- 605 IL
- 488 IN
- 451 US
- 374 CN
5 minutes during peak
- 204991 IN
- 1923 CN
- 1066 TH
- 476 VN
- 472 IL
Across entire 50 min period
- 718633 IN
- 17230 VN
- 16093 CN
- 10238 TH
- 5027 CA
- 397464 - 2405:200::/29 IN - Reliance Jio Infocomm Limited
- 108502 - 2409:4000::/22 IN - Reliance Jio Infocomm Limited
- 100759 - 2402:3a80::/32 IN - Hutchison Max Telecom Limited
- 90870 - 2402:8100::/32 IN - Idea Cellular Limited
- 20171 - 2401:4900::/32 IN - Bharti Airtel Limited