We experienced a significant system clock issue on several Debian virtual machines using systemd-timesyncd.
The incident occurred between 2026-10-08 02:26:51 CEST and 2026-10-08 04:42:53 CEST, corresponding to 2026-10-08 00:26:51 UTC and 2026-10-08 02:42:53 UTC.
Environment
Operating system: Debian 14(forky)
Virtualization: QEMU/KVM managed by libvirt
Clock source: kvm-clock
NTP client: systemd-timesyncd
Observed behavior
The affected virtual machines experienced an apparent forward clock jump of approximately 33 days. The machines then operated with the incorrect future date for approximately two hours and sixteen minutes.
Afterward, systemd-timesyncd corrected the clock backward by exactly:
-2851200 seconds
This is equal to:
-33 days
Our clock-jump monitor reported:
JUMP -2851200.0s at epoch 1791427373; ntp_synced=yes
The correction occurred at approximately:
2026-10-08 02:42:53 UTC
2026-10-08 04:42:53 CEST
Journal evidence
The journal contains the following messages:
Nov 10 00:26:34 systemd-resolvedClock change detected. Flushing caches.
Oct 08 02:42:36 systemd-resolvedClock change detected. Flushing caches.
Oct 08 02:42:36 systemd-journaldTime jumped backwards, rotating.
NTP config
All of the machines which were affected were synced against the following IP 152.53.239.75
Could you please check if there was everything ok with the NTP server?
Yes, this can happen. The pool detected the problem and removed the server from the pool, but unfortunately it was too late for you.
The usual suggestion is to use multiple NTP servers and let the timekeeping software detect any falsetickers. However, systemd-timesyncd might not support such configuration.
My suggestion would be to set up a separate NTP server on your LAN. This server could run chrony, ntpd or any other NTP server software. This server should have multiple upstream servers configured. Pool NTP servers can be used as upstream servers in this situation. If some upstream server acts up, your NTP server would detect this and continue serving correct time, based on other configured time sources.
Your virtual machines should then be configured to query this NTP server of yours for the time. In addition to making sure your virtual machines would get correct time, this would also lessen the load on the NTP pool servers. Instead of all your virtual machines querying the pool for the time, only your own local NTP server would query the pool for the time.
If you are so inclined, you could also set up a local stratum 1 NTP server which gets its time from GPS signal. But for the most common use cases, getting the time from multiple upstream NTP servers would suffice.
I agree with the suggestions. But I also think that perhaps the first course of action should be to simply replace systemd-timesyncd by some sensible NTP client such as chronyd, as it likely is the fastest and easiest and cheapest solution, and in many cases fully sufficient.
It is sad to see that major distributions such as Debian still rely on such inadequate timekeeping software as systemd-timesyncd as their default. Glad to see that at least Ubuntu moved away from systemd-timesyncd, though apparently not primarily for its insufficiencies regarding timekeeping performance, but for its lack of support for NTS.
One argument for systemd-timesyncd often is that something more sophisticated is not needed in many cases. Which certainly is true. But when something more robust and with significantly better timekeeping performance is readily available, with pretty much no downsides, what is the benefit of not using that, except in very resource-constrained environments?
The only thing to add is that when the NTP had a huge offset, the stratum went to 12.
Clients (timesyncd or otherwise) should not sync to stratum 12 sources.
Systemd can’t synch system-host-time from remote in a VPS.
Disable systemd-timesyncd and install Chrony.
But Chrony will complain too.
You have to edit /etc/default/chrony and edit to the following:
# Options to pass to chrony.
DAEMON_OPTS="-F 1"
This -F 1 is a systemcall filter that will keep Chrony from crashing when it tries to access the system-clock of the host, what is not allowed inside a virtual-machine.
With the filter enabled it will keep running and only set the virtual-system-time correct.
Don’t forget the change the /etc/chrony/chrony.conf for correct pool’s/servers.
Maybe use some more trustworthy server of your parent company, I mean they also still run a few atomic clocks contributing to UTC for their legacy time and frequency needs in SONET/SDH networks They should know what time it is in their network.
Otherwise run chrony or ntpd and use more than one server (preferably 3, 5 or 7) to rule out false tickers.
They should know what time it is in their network.