# NTP (GPSd) : Back to 2002 at Oct 24th? **URL:** \ **Category:** Server operators\ **Created:** [September 23, 2021, 6:15am UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159 "2021-09-23T06:15:01Z")\ **Posts on this page:** 19\ **Page:** 1
**Author:** ![marco.davids](https://sea2.discourse-cdn.com/flex016/user_avatar/community.ntppool.org/marco.davids/32/767_2.png) [@marco.davids](https://community.ntppool.org/u/marco.davids)\ **Post date:** [September 23, 2021, 6:15am UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/1 "2021-09-23T06:15:01Z")
[not sure if this was posted before - if it was, please forgive me] TL,DR : from Oct 24, 2021, some NTP servers could suffer a 19 years backwards time warp and go back in 2002. This is due to a bug in the “gpsd” daemon. The bug resides in the gpsd\_gpstime\_resolv() function in gpsd v3.20 to v3.22. It will lead to the function providing a backward time of 1024 weeks (that is about 19 years) on 24 October 2021. More info : > **[A GPSD time warp \[LWN.net\]](https://lwn.net/Articles/865044/)** > > The GPSD project provides a > daemon for communicating with various GPS devices in order to retrieve the > location information that those sensors provide. But the GPS satellites > also provide highly accurate time information that GPSD can > extract for... > **[GPSD time will jump back 1024 weeks at after week=2180 (23-October-2021)...](https://gitlab.com/gpsd/gpsd/-/issues/144)** > > I just discovered a lurking problem in the timebase.c module in all of the branches for releases \>=3.20: In gpsd\_gpstime\_resolv(): The bug is fixed in version 3.23 of gpsd that was released in early August. Interesting background article here: > **[Leap seconds: Causing Bugs Even When They Don't Happen - Bert Hubert's writings](https://berthub.eu/articles/posts/leapseconds-expose-bugs-even-when-they-dont-happen/)** > > Leap seconds are controversial things. Since the Earth does not rotate at a steady rate, over time the Earth could get ahead or behind “atomic time”. Whatever solution you propose for this, someone is going to be unhappy. > I take no position on what... – Marco ---
**Author:** ![alica](https://sea2.discourse-cdn.com/flex016/user_avatar/community.ntppool.org/alica/32/337_2.png) [@alica](https://community.ntppool.org/u/alica)\ **Post date:** [September 24, 2021, 8:46am UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/2 "2021-09-24T08:46:02Z")
Upgraded my gpsd instance from 3.22 to 3.23 this week without noticing the bug… The announcement from gpsd team is lacking important information: the [gpsd-announce post](https://lists.nongnu.org/archive/html/gpsd-announce/2021-08/msg00000.html) did not mention the bug at all, and the [NEWS file](https://gpsd.io/NEWS) only described the bug in an easily-to-be-ignored line. This bug deserves more attention. ---
**Author:** ![Bas](https://sea2.discourse-cdn.com/flex016/user_avatar/community.ntppool.org/bas/32/465_2.png) [@Bas](https://community.ntppool.org/u/Bas)\ **Post date:** [September 24, 2021, 4:22pm UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/3 "2021-09-24T16:22:23Z")
Running 3.17, so no problems to be expected 🙂 ---
**Author:** ![richard](https://sea2.discourse-cdn.com/flex016/user_avatar/community.ntppool.org/richard/32/1473_2.png) [@richard](https://community.ntppool.org/u/richard)\ **Post date:** [September 25, 2021, 1:16am UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/4 "2021-09-25T01:16:31Z")
Thanks for the alert… also running 3.17 (possibly the default on Raspbian?). ---
**Author:** ![Bas](https://sea2.discourse-cdn.com/flex016/user_avatar/community.ntppool.org/bas/32/465_2.png) [@Bas](https://community.ntppool.org/u/Bas)\ **Post date:** [September 28, 2021, 3:48pm UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/5 "2021-09-28T15:48:00Z")
Yes. It is. Raspbian is a bit behind, but that is not a bad thing. ---
**Author:** ![Osama.Elshazly](https://sea2.discourse-cdn.com/flex016/user_avatar/community.ntppool.org/osama.elshazly/32/838_2.png) [@Osama.Elshazly](https://community.ntppool.org/u/Osama.Elshazly)\ **Post date:** [October 20, 2021, 12:36pm UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/6 "2021-10-20T12:36:52Z")
if one of our rhel ntp pool is syncing from a GPSS infected upstream server will this affect our ntp downstream servers ---
**Author:** ![ebahapo](https://sea2.discourse-cdn.com/flex016/user_avatar/community.ntppool.org/ebahapo/32/1152_2.png) [@ebahapo](https://community.ntppool.org/u/ebahapo)\ **Post date:** [October 20, 2021, 6:57pm UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/7 "2021-10-20T18:57:53Z")
Indeed, it could cascade down to other servers, potentially even stratum 0 ones with a good `gpsd`. Probably a good time to add more redundant servers. ---
**Author:** ![Bas](https://sea2.discourse-cdn.com/flex016/user_avatar/community.ntppool.org/bas/32/465_2.png) [@Bas](https://community.ntppool.org/u/Bas)\ **Post date:** [October 21, 2021, 5:19pm UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/8 "2021-10-21T17:19:30Z")
> [@ebahapo](#): > > Indeed, it could cascade down to other servers, potentially even stratum 0 ones with a good `gpsd` . Probably a good time to add more redundant servers. No you can’t, as typical NTP or Chrony use more then 1 source to set correct time. It will ignore infected servers and stop giving time if it’s unsure. ---
**Author:** ![ebahapo](https://sea2.discourse-cdn.com/flex016/user_avatar/community.ntppool.org/ebahapo/32/1152_2.png) [@ebahapo](https://community.ntppool.org/u/ebahapo)\ **Post date:** [October 21, 2021, 6:50pm UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/9 "2021-10-21T18:50:00Z")
Unless infected servers are the majority peers. And if pool servers stop serving time, they degrade the reliability of the pool. ---
**Author:** ![yudi.dudulz](https://avatars.discourse-cdn.com/v4/letter/y/82dd89/32.png) [@yudi.dudulz](https://community.ntppool.org/u/yudi.dudulz)\ **Post date:** [October 22, 2021, 5:00am UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/10 "2021-10-22T05:00:15Z")
@Bas Friend, can you help check whether pool ntp org or asian pool ntp org is not running in this version? **3.20 to 3.22** ---
**Author:** ![alica](https://sea2.discourse-cdn.com/flex016/user_avatar/community.ntppool.org/alica/32/337_2.png) [@alica](https://community.ntppool.org/u/alica)\ **Post date:** [October 23, 2021, 3:37pm UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/11 "2021-10-23T15:37:34Z")
> [@yudi.dudulz](#): > > can you help check whether pool ntp org or asian pool ntp org is not running in this version? Generally this would not be possible, as default GPSD settings shall listen on loopback interface and cannot be detected from internet. After all the bug shall be triggered in hours, we will find the result soon… ---
**Author:** ![mlichvar](https://avatars.discourse-cdn.com/v4/letter/m/e79b87/32.png) [@mlichvar](https://community.ntppool.org/u/mlichvar)\ **Post date:** [October 23, 2021, 4:16pm UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/12 "2021-10-23T16:16:35Z")
I think some servers could ignore a large jump of its reference clock, falling back to NTP servers, but fail later after restart if the refclock is selected before it is compared to other sources. This depends on the configured polling intervals and iburst. I guess will we see in the monitoring data some servers changing stratum from 1 to 2, but hopefully the number of servers jumping back to 2002 will be very small. ---
**Author:** ![alanw](https://avatars.discourse-cdn.com/v4/letter/a/ebca7d/32.png) [@alanw](https://community.ntppool.org/u/alanw)\ **Post date:** [October 23, 2021, 11:08pm UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/13 "2021-10-23T23:08:35Z")
I have some old stats running somewhere for the UK pool. I have the following counts for IPv4 only. - Score is \>10 - Not scheduled for deletion | stratum | count | | --- | --- | | 1 | 43 | | 2 | 92 | | 3 | 46 | | 4 | 5 | | 5 | 1 | It will be interesting to see tomorrow if that looks much different. ---
**Author:** ![mlichvar](https://avatars.discourse-cdn.com/v4/letter/m/e79b87/32.png) [@mlichvar](https://community.ntppool.org/u/mlichvar)\ **Post date:** [October 24, 2021, 8:39am UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/14 "2021-10-24T08:39:18Z")
In my logs I see 5 servers that changed stratum from 1 to 2 after the midnight. I don’t see any large offsets. There is a large number of servers that I didn’t get any response from yet, but I think that’s just the well-known NTP rate limiting of the Internet. ---
**Author:** ![alanw](https://avatars.discourse-cdn.com/v4/letter/a/ebca7d/32.png) [@alanw](https://community.ntppool.org/u/alanw)\ **Post date:** [October 24, 2021, 8:58am UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/15 "2021-10-24T08:58:21Z")
I’ve only see two hosts in the UK change from stratum 1 to stratum 2. | stratum | count | | --- | --- | | 1 | 41 | | 2 | 95 | | 3 | 44 | | 4 | 6 | | 5 | 1 | ---
**Author:** ![stevesommars](https://sea2.discourse-cdn.com/flex016/user_avatar/community.ntppool.org/stevesommars/32/2075_2.png) [@stevesommars](https://community.ntppool.org/u/stevesommars)\ **Post date:** [October 26, 2021, 3:08am UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/16 "2021-10-26T03:08:15Z")
I found two NTP Pool servers that might have been affected, one in the US and one in the UK. ---
**Author:** ![ebahapo](https://sea2.discourse-cdn.com/flex016/user_avatar/community.ntppool.org/ebahapo/32/1152_2.png) [@ebahapo](https://community.ntppool.org/u/ebahapo)\ **Post date:** [October 26, 2021, 7:29pm UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/17 "2021-10-26T19:29:56Z")
I guess that the outcome was that there were some outlier cases, but the `gpsd` issue didn’t really cause a dent in the pool. Thanks in great part to NTP clock selection algorithms and their implementation in `ntpd` and `chronyd`, and to the pool selection mechanism. ---
**Author:** ![Bas](https://sea2.discourse-cdn.com/flex016/user_avatar/community.ntppool.org/bas/32/465_2.png) [@Bas](https://community.ntppool.org/u/Bas)\ **Post date:** [October 27, 2021, 3:29pm UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/18 "2021-10-27T15:29:26Z")
> [@yudi.dudulz](#): > > @Bas Friend, can you help check whether pool ntp org or asian pool ntp org is not running in this version? **3.20 to 3.22** There is no need to check it as servers that give bad time are ignored by default. And no, you can only check this on the server that give out time. In general you do not need to worry. Typical servers use more then 1 source and they check. I run my stratum 1, but it also compares about 6 other servers, when my GPS is wrong it will use one of the other sources are correct time. Stratum2 servers are supposed to use 3~6 other servers for correct time too. It will not matter if a few are not correct, the system will know and ignore them. By the time it reaches the pool all troubles are already filtered out. The pool also checks the correct time, else they will not be served. I run chronyd, that is far more smart then ntpd. ---
**Author:** ![alanw](https://avatars.discourse-cdn.com/v4/letter/a/ebca7d/32.png) [@alanw](https://community.ntppool.org/u/alanw)\ **Post date:** [October 27, 2021, 11:40pm UTC](https://community.ntppool.org/t/ntp-gpsd-back-to-2002-at-oct-24th/2159/19 "2021-10-27T23:40:46Z")
Many thanks to @marco.davids for mentioning the issue, and, with plenty of advance notice.