Why is Germany using server from AT pool?

Dear All,

I am running three stratum-2 server in AT-pool with IPv6. These three server cover 20% of all Austrian IPv6 server which are total 15.
These days I verified geolocations where my servers communicate. Germany is a daily hotspot with regular NTP traffic.

I ask myself why ? Germany has itself 300 IPv6 server. I don’t see a reason to use the Austrian’s too.

Looking at the map there are some blue colored areas around the world. But these are most attacks, especially from India.

I would highly appreciate not to be in the German pool too.

Kind regards
Hans

1 Like

Hi @HansMayer – it’d be easier to debug if you had a client in Germany that was being sent to an Austrian server!

Most likely what’s happening is that some users in Germany are using a name server that’s in Austria (or just listed in the geoip database as being in Austria).

Hi Ask,

Many thanks for your fast feedback.

Unfortunately it’s not only “some” user. When I select “Germany” - and of course I have to trust what my netflow analyses tool tells me about Germany - I see for each of my NTP server about 32 Kbit/sec well distributed over all 3 NTP server all over the day incoming traffic. The same for outgoing. It tells me also there are 272264 clients from Germany. And there are also 11049405 flows within 24h coming from DE which fits quite well with the average traffic I can see.

Just checking a few entries with "ntpq -c mrul’ I see also a lot of German domains. And of course the list is endless and takes endless time. But both information (ntpq and netflow) fit together.

Definitely we do not speak about “some”.

I tried to query several times “2.de.pool.ntp.org” because IPv6 is only in the “second” pool but I never have seen my IP addresses. But this is maybe not possible within Austria.

I don’t think that the GeoIP DB is wrong as reverse lookup of several IPv6 addresses end with .de too.

Is there any other information I can provide to you to convince you that at least my servers are used intensively from Germany.

Kind regards
Hans

Maybe it’s some mis-located IP prefix, possibly belonging to Liberty Global (which is owning UPC Austria and Unitymedia/KabelBW in Germany).

Could you provide one IP address of your affected servers?
I would do a RIPE Atlas DNS measurement on the major provider networks in Germany and see, on which providers it comes up.

Hi,

Many thanks for your offer to do a measurement.
These are my NTP servers:

entp1.iiasa.ac.at. 10800 IN AAAA 2001:628:21f0:80::80:35
entp2.iiasa.ac.at. 86400 IN AAAA 2001:628:21f0:80::80:160
entp3.iiasa.ac.at. 10800 IN AAAA 2001:628:21f0:80::80:29

How can you look for these settings with a RIPE Atlas. We have also one installed. It’s Probe #35603

Kind regards
Hans

I could narrow down this issue a little bit.
One of the networks is 2001:16b8::/32 with domain “versatel-1u1.de
But there are many others too.

// Hans

I made a measurement a few minutes ago with ~750 probes participating.
None of these got an answer even starting with “2001:628” :frowning:
The complete list is available here: https://maxderdepp.de/files/npt-atpool-atlas-2.txt
There should be a lot of probes in AS8882 (Versatel), so you might get information about the DNS resolvers used.

I configured the measurement to query for 2.de.pool.ntp.org, type AAAA using the probe’s resolver and spread the measurement over 240 seconds.

Have you done some investigation on the raw traffic (i.e. analysed from sflow, aggregated to /32 prefixes), just to make sure there is really lots of traffic originating from germany?
Maybe it’s not the pool’s DNS infrastructure but your local GeoIP database, which is mislocating the requests :wink:

The “de” sub zone would never return an “at” server.

What happens (as I think you mentioned), is that users in Germany are using a DNS server that the system thinks is in Austria. The system only gets the IP of the DNS server, not the NTP client IP.

https://www.mapper.ntppool.org/ is a debug tool that checks the IP of your DNS resolver.

https://www.mapper.ntppool.org/json returns the data, for example:

curl -Ls https://www.mapper.ntppool.org/json

Of course that doesn’t help find miss located servers unless it’s happens to be the one you use…

Given more subnets of German clients hitting an Austrian server and with enough “probes” to the mapper there might be data to figure it out.

A txt query like this gives some debug data, too. Maybe that’s something RIPE Atlas can return for a bunch of probes:

dig -t txt _country.pool.ntp.org
1 Like

Dear All,
thanks for coming back to my question.

I was looking a bit closed who is in my “reslist” of ntpq. Below there are the top-10 of DE domains. The number in first column gives to count of seen sub-domains. “Winner” is t-ipconnect.de with more than 50000 different sub-domains I have seen. And of course much more ntp request.

It seems this is a provider and they configured the DSL modems for their customers wrong. But they are not alone. There are several others too.

Is there something what we can do ?

// Hans

52761 t-ipconnect.de
791 net-htp.de
332 vodafone-ip.de
177 netcologne.de
74 kabel-deutschland.de
56 dg-w.de
38 ewe-ip-backbone.de
20 kabel-badenwuerttemberg.de
19 unitymediagroup.de
16 encoline.de

I would be happy to set up a RIPE Atlas measurement but I am not sure on how useful that txt record is as it does not appear to return the country that it thinks you are in. It lists some IP addresses but they are weird. The IPv4 ones are RFC1918 space but not what I use internally and I don’t have a clue about the IPv6, perhaps you could let me know what it means.
"192.168.100.2:56804" "2001:569:2::" "@" "/0" "192.168.100.2" "192.168.100.2"

No, that was actually useful. There are some of the dns servers running an older version of GeoDNS with more limited IPv6 support. We will figure out to get it upgraded or maybe temporarily disabled. (Cc @gfk)

I’ve set up a measurement and I will put up the results when it is finished. Most of the answers look like the following (this is a sample)
["173.194.169.78:36746","185.240.52.0","de europe @","/0","46.227.203.69","46.227.203.69","()"]

Measurement is here: https://atlas.ripe.net/measurements/20733520/
500 probes from Germany. There is a JSON download. I don’t have time to analyze at the moment but I would be interested if any do not show up as in the germany zone.

I have over 7 million credits on the ripe atlas site if you want me to transfer some to you for running more probe tests.

I’ve got 15 million from my three probes so I am good. I’m not entirely sure what, if any, other tests need to be run.

Dear All,

many thanks for all your response. Bryce, your measurement is impressive looking at the map.
What can be done now ?

// Hans

From your measurements, 3 of the probes used a server that the NTP Pool servers thinks are in Austria:

$ cat RIPE-Atlas-measurement-20733520.json  \
   | jq ' .[].resultset | .[] | select(.result.answers != null) 
          | .result.answers[] | .RDATA  ' \
   | grep europe | sort | uniq -c
   3   "at europe @",
   2   "cz europe @",
 762   "de europe @",
   1   "eu europe @",
   2   "fr europe @",
   2   "it europe @",
   1   "lu europe @",

Another data point, that didn’t give much – from https://www.mapper.ntppool.org/ I don’t see any relevant seeming resolvers from Austria showing up with German clients. Though there isn’t much data. If you have a website popular with users in Germany and Austria, it’d be helpful if you could add https://www.mapper.ntppool.org/mapper.js on it. :slight_smile: (There are non-javascript versions, too).

select server_ip, server_asn, count(*) from ips where client_cc = 'DE' and server_cc = 'AT' and last_seen > '2018-01-01' group by 1,2 order by count(*) desc limit 8;
    server_ip    | server_asn | count
-----------------+------------+-------
 37.235.1.174    |      51453 |    43
 37.235.1.177    |      51453 |    34
 66.185.117.242  |         42 |    16
 213.33.99.87    |       8447 |    15
 194.0.230.25    |      42783 |    13
 66.185.117.244  |         42 |    13
 66.185.117.243  |         42 |    12
 213.33.99.76    |       8447 |    11
 213.150.228.38  |      42587 |    10

Dear All,

Just to make sure, I am only speaking about IPv6. Because I see very often in the postings above only IPv4. These 3 NTP server I am running are only registered with IPv6 in AT pool.

in Austria : 3 “at europe @”,

If these 3 probes are using the same DNS resolver as my top list than we know.

Since about 18 days I registered about 17000000 queries from 2003::/19 for only one of my NTP server. 2003::/19 belongs to DE-TELEKOM

// Hans

I recently discovered, that all Fritzboxes(very popular Router in Germany) has the default NTP Server setting of 2.europe.pool.ntp.org, which is a ipv6 enabled record. Yes, they are not using vendor-zones, which is a different issue, I will write them an Mail about it.

But my best guess is, that these(and probably a lot of other) are devices querying europe and getting your server. And since many Internet Connections in Germany are IPv6 enabled, a lot of them query v6 Servers.