Issue with registration of a test NTP Pool Agent version v4.1.4

Monitoring Operators
,
1

My issue is on aarch64, Raspberry PI 4 with Tumbleweed as OS, but I do not think that matters.

Seting-up a new test monitor, sudo journalctl -u ntppool-agent@\* -f gives:

Apr 24 08:44:40 tumbleweed.home ntppool-agent[51010]: level=WARN msg="no API key, please run ntppool-agent setup" env=test cmd="ntppool-agent setup --env test --state-dir '/var/lib/ntppool-agent'" wait_time=4m59s
Apr 24 08:44:43 tumbleweed.home ntppool-agent[51010]: level=WARN msg="failed to refresh JWT token" env=test appconfig-manager.err="no API key available for JWT token request"

and that is normal. However, the next step
sudo -u ntpmon ntppool-agent setup -e test -a 355n9ds
gives a 404 error:

tumbleweed:~ # sudo -u ntpmon ntppool-agent setup -e test -a 355n9ds
time=2026-04-24T08:01:30.440Z level=INFO msg="using hostname for registration" env=test hostname=tumbleweed.home
default backend - 404
tumbleweed:~ #
2

This message from a configured test monitor may be related:

level=WARN msg=“failed to reload AppConfig” env=test appconfig-manager.err=“unexpected response code: 404 (trace )”

Other than that message my test monitors seem to be working fine.. Oh, or maybe not, because I also get:

Apr 24 09:29:53 level=INFO msg=“certificate expiring, request renewal” env=test nextCheck=2h0m0s
Apr 24 09:31:54 level=INFO msg=“certificate expiring, request renewal” env=test nextCheck=2h0m0s
Apr 24 09:33:54 level=INFO msg=“certificate expiring, request renewal” env=test nextCheck=2h0m0s
Apr 24 09:35:54 level=INFO msg=“certificate expiring, request renewal” env=test nextCheck=2h0m0s
Apr 24 09:37:55 level=INFO msg=“certificate expiring, request renewal” env=test nextCheck=2h0m0s

When the certificate actually expires in around 1h 20m the agent might start having problems communicating. We’ll see. But yes, there are issues, it seems.

openssl x509 -in /var/lib/ntppool-agent/test/cert.pem -text -noout | grep Not
Not Before: Apr 19 11:03:34 2026 GMT
Not After : Apr 24 11:04:04 2026 GMT
date
Fri Apr 24 09:44:01 UTC 2026

3

Looks like (re)starting the agent does not work at the moment:

systemd[1]: Started ntppool-agent@test.service - NTP Pool Monitor (test).
ntppool-agent: level=INFO msg="certificate expiring, request renewal" env=test nextCheck=2h0m0s
ntppool-agent: level=INFO msg="load failed" env=test err="unexpected response code: 404 (trace )"
ntppool-agent: Usage: ntppool-agent monitor --env=test [flags]
ntppool-agent: run monitor
ntppool-agent: Flags:
ntppool-agent:  -h, --help                Show context-sensitive help.
ntppool-agent:      --debug               Enable debug logging
ntppool-agent:      --log-level="info"    Log level for stderr (debug, info, warn, error)
ntppool-agent:                            ($MONITOR_LOG_LEVEL)
ntppool-agent:      --state-dir=STRING    Directory for storing state ($MONITOR_STATE_DIR)
ntppool-agent:  -e, --env=test            Deployment environment (prod, test, devel)
ntppool-agent:                            ($DEPLOYMENT_MODE)
ntppool-agent:      --[no-]ipv4           IPv4 monitor (default)
ntppool-agent:      --[no-]ipv6           IPv6 monitor (default)
ntppool-agent:      --once                Only run once instead of forever
ntppool-agent:      --sanity-only         Only run the local sanity check
ntppool-agent: ntppool-agent: error: unexpected response code: 404 (trace )
systemd[1]: ntppool-agent@test.service: Main process exited, code=exited, status=1/FAILURE
systemd[1]: ntppool-agent@test.service: Failed with result 'exit-code'.

This seems to affect only the test environment so it’s not the end of the world, but it’s still worth fixing, of course.

4

Most of my test monitors are being reported as being disconnected since 2026-04-22. Only two were still connected. Just restarted one of them as part of updating the ntpool-agent package, and it’s now gone as well.

So I guess as long as a back-end connection was still active since before the current outage on beta, it’ll keep working. But once it needs to be re-established, e.g., due to agent restart, or newly established (as during registration), the daemon will refuse to run due to the inability to successufully connect with the back-end.

It will be interesting to see whether the upcoming expiry of the certificate will also break the connection. I guess as long as it tries to obtain a new certificate over an existing connection, it might be fine. But should it get a new certificate and try to re-establish the connection, that might break things.

Except that it seems not unlikely that once the certificate renewal starts working again, also the other issue currently preventing connections would be solved as well - assuming both have the same underlying likely cause of systems not yet being properly interconnected again after the recent migration of the beta clusters (and many other things before that) to new infrastructure.

5

Repeating much of what MagicNTP wrote above, but as I had this message already prepared I’ll paste it here anyway:

Ho hum, nothing happened when the certificate expired. I had left this test agent instance running, waiting for the cert to expire (the server where I tried to restart the agent was a different one). So this instance will keep testing for now.

But I get it, the agent keeps a connection open all the time and the cert gets checked only when establishing the connection. Restarting the service would probably trigger a certificate check. But restarting the service would currently fail anyway due to the issue mentioned above.

Restarting the agent that was already failing to start up (my previous message) but with a certificate that had expired in the meantime gave me this:

level=INFO msg=“certificate expiring, request renewal” env=test nextCheck=2h0m0s
level=INFO msg=“load failed” env=test err=“unexpected response code: 404 (trace )”

I had expected some other error than a 404. Maybe renewing the certificate does not need a valid certificate?