GPS rollover may malfunction on or after April 6

#1

Dears,
Any idea about the impact of this rollover on the ntp pool time?
Kind regards,
Dirk

0 Likes

#2

The expected impact is low.

If any servers starts serving the wrong time, they will quickly be removed from the system.

Most clients should be recognizing the clearly bad time (very different from the other signals they get, before their manufacturing date, etc) and ignore the signal.

Of course a server can serve bad time for some minutes before it gets noticed by the monitoring system and DNS updated and clients might not be well implemented.

0 Likes

#3

Dirk, the majority of implementations only allow the time to be changed gradually unless otherwise specified. That goes for most stratum two and above out there. I suppose stratum one may be the most affected but they will fix themselves if so and if they do not, then what ask said will apply. I would guess non ntp.org code base would be more affected but I guess you will find out if you purchased the right car!

0 Likes

#4

Because this isn’t the first time the GPS week has rolled over, a large number of GPS modules already handle this by making sure that every date is past a reference timestamp (usually firmware build date). There are some modules that have a command to tell it what date to use as its reference date (such as Skytraq’s “CONFIGURE UTC REFERENCE TIME SYNC TO GPS TIME”)

Because GPS time is internally consistent, even if the GPS module doesn’t have the right time, its PPS should be on time. So in that case, stratum 1s can combine an external source of time with the local PPS to get the correct time.

More info:

0 Likes

#5

Is there an official statement written by principals at the NTP Pool to post (Please)?

I have been getting quite a few inquiries asking NTF to,
“promise that the ntp.pool.org is not effected during the incident.”

I can direct them to the NTP Pool, but as per usual, some folks erroneously assume that Network Time Foundation IS hosting SERVERS in the POOL. I’d like to put the link to this discourse and a comment from Harlan Stenn in this post also.

Anyone on this list have the authority to speak on behalf of the NTP Pool on this topic?

0 Likes

#6

I believe it’s affected, not effected… The GPS rollover effect could affect a poorly configured NTP server.

Admit nothing, deny everything, make counter accusations.

The NTP Pool monitor should catch any servers with the incorrect time and drop them out of the pool. That would be the definitive answer I would think.

Without knowing every last piece of hardware that is out there, you can’t really make any promise.

Assuming someone has a GPS that won’t rollover properly, and it’s the only source configured on a NTP server, then NTPD would reject it and mark it as a falseticker because of the huge jump. NTPD would either stop serving time, or if they configured their local clock as a high-stratum it would simply freewheel. Once it drifted too much then again the pool monitor would drop it from the pool.

If somehow it did rollback and decide it was going to party like it was 1999, and somehow people were using that source, then clients running an actual NTP build with multiple sources would simply reject that source as a falseticker. If some client was only querying one source with like ntpdate, then it’s up in the air. If they coded in at least a basic check to ensure time doesn’t go backwards, or at least can’t be before a certain date (usually a build date or BIOS date), then it should try another source. However, seeing how poorly some of these companies code… Ehhh…

It would be kind of tedious circumventing built-in protection schemes in NTPD using various flags no normal person would use under any regular circumstances, and take an improper configuration for it to rollback and not crap on itself.

Caveat Utilitor… (Let the user beware…)

0 Likes

#7

As I wrote above, it’s possible that some servers briefly will serve a wildly inaccurate time (as it always is), but they will quickly be weeded out of the system and a properly implemented client will not be affected.

0 Likes

#8

I just noticed there is a firmware 4.20 for the Garmin 18x that includes a fix for the rollover. It seems it was released only two weeks ago! I guess better late than never. It used to be a popular cheap receiver used on NTP servers and I’m sure there is still a number of them in the pool. I have one here that’s currently unused. I think I’ll leave it unpatched to see what happens.

The firmware can be downloaded here:
https://www8.garmin.com/support/download_details.jsp?id=4055

0 Likes

#9

According to Garmin’s statement, these devices should be unaffected right now… Most devices will hard code firmware built date as its epoch, so a GPS device with 2010-built firmware will likely treat GPS week 0 as from 2019, not 1999. If this device lasts to 2030 without any firmware update (lack of user action or vendor vanished), then the rollover will be a problem.

0 Likes

#10

Assuming the event was midnight April 7 (45 minutes ago) then the monitoring system detected no servers that jumped dramatically in time at that time.

BigQuery query:

select server_id, ts, score, offset, error from ntppool.log_scores
    where abs(offset) > 1
        and ts > '2019-04-06 22:00:00'
        and ts < '2019-04-07 02:00:00'
        and monitor_id > 0
       and (leap != 3 or leap is null)
    order by id;

None of the servers with a larger than 1 second offset in the last couple hours were active in the pool (their score was already low).

If you remove the leap conditions there are a few servers that returned “not synchronized” because of rate limiting.

The form at https://www.ntppool.org/scores/ takes a server ID and redirects to the server page (or you can just use the ID in place of the IP).

0 Likes

#11

In my logs all I can see is just one server that changed its stratum from 1 to 2 on the rollover and one that switched to 2 and few minutes later back to 1 about 2 hours after the rollover.

We may see more servers having issues later when they are power cycled.

My Garmin 18x was reporting correct date after the rollover, but only until it was power cycled. It now reports 1999-08-22. I’ll see if the new firmware fixes that. Two u-blox receivers I was monitoring are ok.

0 Likes

#12

Ah, that’s a good point. Those could in principle get turned on and think they are in sync.

It’s interesting how an unattended or minimally maintained stratum 2 installation in many cases will be more robust than a stratum 1 setup.

0 Likes

#13

Right. And if you have an older GPS receiver this may happen at the end of any week. See also:
https://kb.meinbergglobal.com/kb/time_sync/gnss_systems/gps_week_number_rollover#a_simple_error_prone_approach

0 Likes