Intention to enable IPv6 by default in 2017

Announcements
43

Hi Bas,

(translation: “Why does your test work, but my own DNS doesn’t?”)

I don’t know for sure - but I am pretty sure it is something trivial. I still suspect it could be the DNS rebinding protection that I mentioned earlier. That’s because ‘ntp5.heppen.be’ resolves to an address within your 2a02:578:440e::/48 range (with one exception, see below) and the DNS rebinding protection built into your router won’t allow that, because it is a security risk. Did you try to add ‘ntp5.heppen.be’ to the exception list as well?

It may seem that this situation does not exist for IPv4, but that isn’t the case. For IPv4 you use NAT and you don’t have any DNS records within your 192.168.1.0/24 range, because that doesn’t make sense for a machine that has to be reachable from the internet. But if you would do that, than you’d probably see the exact same behaviour for IPv4: rebind protection. I created a little test below to show it.

The only part that isn’t clear to me yet, is why your 127.0.0.53 does resolve the IPv6 address of ‘ntp5.heppen.be’. My guess is you are running this query on a box with ‘systemd-resolved’ enabled that doesn’t use 192.168.1.1 (your router) as the resolver, but I could be wrong.

By the way; are you aware that the three authoritative name servers for ‘heppen.be’ are not in sync? The SOA serial of ‘christina.neostrada.nl’ is running behind. And ‘ns.heppen.be’ is serving another IPv6-address for ‘ntp5.heppen.be’ (2001:41d0:203:654d::8f37:5630) than the other two (2a02:578:440e:0:5f05:fa04:1720:d90f). And only ‘ns.heppen.be’ serves an IPv4 address (5.135.125.103).

There is also a discrepancy between the NS set of ‘heppen.be’ in the child (‘heppen.be’ zone) and the parent (‘be’ zone). Meaning that ‘heppen.be’ is not configured correctly in (the) DNS. See this Zonemaster output. And this link shows it as well, but it might be harder to read.

Last but not least; I recommend to use ‘dig’ for DNS queries. It gives you much more control over what exactly you are trying to do. Much like the ‘set debug’ option of nslookup, but better. And it has lot’s of options to help with debugging DNS.

For example, try this:

dig +nssearch heppen.be

You will quickly spot the difference in SOA serial number.

Hope this helps!

Test:
I setup a small test to try DNS rebinding protection. It is designed for your specific situation: 192.168.1.0/24 and 2a02:578:440e::/48. And it won’t work anymore if these ranges change.

Try to do this:
nslookup rebind.testdns.nl 192.168.1.1

It won’t work.

Now try this:
nslookup rebind.testdns.nl 1.1.1.1

It will work

Now add ‘rebind.testdns.nl’ to your rebind protection exceptions list and run the tests again (but wait a couple of minutes to let this change come into effect).

They will now both work.

2 Likes
44

I haven’t been keeping up with this thread (mostly I have been focusing on the new monitoring system and there was some unscheduled system maintenance work this fall plus the regular ongoing care and feeding); but I want to interject that the premise that “there’s no IPv6” seems a little false to me.

The system keeps DNS logs for about 36 hours, and in the last ~36 hours the system has handed out ~2.5 billion IPv6 server addresses. This is a lot less than the ~26 billion IPv4 addresses, but a lot of that difference is that we get more A queries (~8 billion) than AAAA ones (~2 billion).

Only 1/3 of the servers in the system are IPv6, so if any of you are wondering if it’s worth adding more – yes, there are a lot of countries with very poor IPv6 coverage.

5 Likes
45

Is there any plan to add IPv6 AAAA records to any of the pool hostnames other than 2.pool.ntp.org?

2 Likes
46

Overall, there may be fewer AAAA queries because the negative responses are more cacheable (the TTL is higher and EDNS Client-Subnet isn’t a factor).

On the other hand, resolvers or clients may cache them less.

And who knows how many clients make AAAA queries but don’t actually use IPv6.

2 Likes
47

Is that so?

I count 3085 IPv4 servers and 1589 IPv6. That is a little over 51% right? Or do you refer to something else and am I misunderstanding?

Also, very interesting to see how 25% of the queries are for an AAAA-record, whereas only 9.62% of the answers are an AAAA. Indeed a very skewed ratio. Imagine how that number could go up if more (all) instances, besides 2.pool.ntp.org, would serve AAAA records. Especially with taking @mnordhoff’s remark about negative caching into account.

1 Like
48

That is 4674 as total server count. Out off that as base; 1589 is 34%, which is about one third.

With other words: 1/3 of the servers in the system are IPv6 and 2/3 of the servers in the system are IPv4.

The interesting part is the following:

That value we have no control over. That covers the population of the NTP clients, of course the value is biased by the above mentioned DNS caching differences.

49

We are probably both right. You assume that the IPv6-servers don’t have an IPv4 address (which is at least true for my personal servers), I was assuming they where mostly dual-stack. We don’t know the exact figures (unless @ask knows them?). It is anywhere between 34% and 51%, leaning more towards the latter, I would say.

Exactly, but we can control the amount of AAAA-responses we return.

2 Likes
50

Yes. It would be interesting to know the ratio of the AAAA queries for the 2.[subdomain.]pool.ntp.org relative to all the AAAA queries. So we could estimate the change when enabling IPv6 for all combinations of the ntppool domain names.

2 Likes
51

It’s to be expected that a third of the addresses are served by a quarter of the host names, 2.pool.nto.org. So the ratio of one every 11.5 addresses being IPv6 indicates that the pool is artificially capping serving such addresses and more host names should have AAAA records so that the addresses handed would reflect the server population in the pool better.

1 Like
52

If the client queries the IPv6 address it’s because it may use them. Some clients query both and use the one returned first. Others, query one first and then the other after not getting a reply.

But the pool cannot control that. What it should do is serve as many IPv6 addresses as there are such servers in the pool, which will not happen until all its hostnames, pool.ntp.org and [0..3].pool.ntp.org, as well as custom subdomais, have AAAA records.

1 Like
53

No, that covers the under representation of the IPv6 servers in the answers due to not all hostnames servers by the pool having the AAAA record.

1 Like
54

The statistics we discuss is about the DNS queries, not about the DNS answers. The DNS queries depend exclusively the NTP client’s host configuration and their network connectivity.

But you are right, the statistics of the answers is dependent on the pool DNS server data configuration as well. (Not all names have associated AAAA records, only 2...pool.ntp.org. That is the main topic of this thread.)

1 Like
55

I’m still doing some data analysis, but a quick first cut of data I collected today. This was 1 Hour (1600 UTC to 1700 UTC). Two small dedicated servers with the exact same hardware, running only chrony and sshd on Ubuntu 22.04.1 LTS. Both are located in the same datacenter (OVH Roubaix).

The IPv6-only server received 3.5% of the total client requests received by the IPv4-only server.

image
image829×623 29.2 KB

56

Interesting stuff, but I don’t quite understand your setup, so please elaborate. For instance, if I look here, I don’t get a confirmation that time-eu02 is IPv6-only.

Are they both configured with the same bandwidth in the NTP pool dashboard?

I would love to hear more details and conclusions of your analysis. So keep up the good work and thanks!

57

Here’s a quick overview of the setup. I’ve done these tests dozens of times over the past few days. They’ve all had very similar results.

Hostname: time-eu01.ntstime.net
IPv4: 178.32.216.31
IPv6: 2001:41d0:8:da1f::1
IPv4 Speed: 500 Mbit
IPv6 Speed: Disabled, monitoring only
Other: Current score: 20.0 | Stratum: 2 | Zones: @ europe fr

Hostname: time-eu02.ntstime.net
IPv4: 5.39.92.42
IPv6: 2001:41d0:8:b12a::1
IPv4 Speed: Disabled, monitoring only
IPv6 Speed: 1000 Mbit
Other: Current score: 20.0 | Stratum: 2 | Zones: @ europe fr

To clarify, time-eu01 had IPv6 disabled in the dashboard, so dual-stack is probably not the right term, but it doesn’t affect the data.

Thank the guy that made that ntptraf2 tool. :grinning: I wouldn’t have been able to see the differences without it.

1 Like
58

Right, right… So this is a comparison, basically, between IPv4-only and IPv6-only.
(keeping in mind also the difference in configured ‘speed’).

Ok, now I get it.

Interesting, thanks (and thank you for the compliment).

I do similar measurements and it is really interesting to see the uptake of IPv6 in Germany, because a major CPE-equipment vendor there (AVM, making Fritz!Box), has 2.[europe].pool.ntp.org configured by default.

1 Like
59

It looks like Ubuntu has done something similar in their default chrony config starting in 2018. These are the comments in the default config in 18.04 through 22.04 LTS:

# This will use (up to):
# - 4 sources from ntp.ubuntu.com which some are ipv6 enabled
# - 2 sources from 2.ubuntu.pool.ntp.org which is ipv6 enabled as well
# - 1 source from [01].ubuntu.pool.ntp.org each (ipv4 only atm)
# This means by default, up to 6 dual-stack and up to 2 additional IPv4-only
# sources will be used.
# At the same time it retains some protection against one of the entries being
# down (compare to just using one of the lines). See (LP: #1754358) for the
# discussion.
1 Like
60

Same here, not much IPv6 requests, dial-stack server:

Schermafdruk op 2022-12-15 16-41-42
Schermafdruk op 2022-12-15 16-41-42814×464 39.1 KB

The question is, if IPv6 is booming, why are the requests not higher?

One answer can be that DNS-caches like in my Fritzbox don’t hand out IPv6 by default.
If many routers do this, it will explain why IPv6 is so far behind on requests.

@marco.davids, I did try a lot of stuff and altered my ns.heppen.be to mach the other DNS-servers (was not ware it’s used but seems to be glued). However, that did not fix anything. Could also be that it’s not listed as DNS-server for my domain. It’s just part of KeyHelp that is turned on because I need to check translations in Dutch :grin:

61

But it does - we proved it with ‘example.nl’ :wink:

1 Like
62

True…but I still don’t get it…

Found it!!!

Thanks a lot Marco, still the Fritzbox is stupid on resolving.
Rebind does work, but get this.

The problem is the Linux resolver-cache. Somehow the router does not tell the client-pc that information has changed.

It simply doesn’t give the new information. How silly is that?
Not even after a restart of Linux. I needed to flush the DNS-caches to make it work.

Is this a bug? I do not know, but I do know that 1.1.1.1 had no problems with any change, the Fritz kept giving cache but not the actual information.

Well it’s found and solved…pffffff that was a tough one.

BTW, I opened another ticket by AVM, hopefully they see the problem, as it does have a problem with the DNS-cache.

1 Like