Issues with api-buzz.mon.ntppool.dev

Monitoring Operators
1

@ask, since a few days I think, the system logs on various of my monitors are getting spammed by messages like the following:

Dec 19 14:01:54 debian ntppool-agent[25478]: level=INFO msg="failed to send logs to https://api-buzz.mon.ntppool.dev/v1/logs: 404 Not Found (body: default backend - 404)"
Dec 19 14:02:04 debian ntppool-agent[25478]: level=INFO msg="failed to send logs to https://api-buzz.mon.ntppool.dev/v1/logs: 404 Not Found (body: default backend - 404)"
Dec 19 14:02:06 debian ntppool-agent[25478]: level=INFO msg="failed to upload metrics: failed to send metrics to https://api-buzz.mon.ntppool.dev/v1/metrics: 404 Not Found (body: default backend - 404)"
Dec 19 14:02:14 debian ntppool-agent[25478]: level=INFO msg="failed to send logs to https://api-buzz.mon.ntppool.dev/v1/logs: 404 Not Found (body: default backend - 404)"
Dec 19 14:02:21 debian ntppool-agent[25478]: level=INFO msg="failed to upload metrics: failed to send metrics to https://api-buzz.mon.ntppool.dev/v1/metrics: 404 Not Found (body: default backend - 404)"
Dec 19 14:02:24 debian ntppool-agent[25478]: level=INFO msg="failed to send logs to https://api-buzz.mon.ntppool.dev/v1/logs: 404 Not Found (body: default backend - 404)"
Dec 19 14:02:33 debian ntppool-agent[25478]: level=INFO msg="traces export: failed to send to https://api-buzz.mon.ntppool.dev/v1/traces: 404 Not Found (body: default backend - 404)"
Dec 19 14:02:34 debian ntppool-agent[25478]: level=INFO msg="failed to send logs to https://api-buzz.mon.ntppool.dev/v1/logs: 404 Not Found (body: default backend - 404)"
Dec 19 14:02:36 debian ntppool-agent[25478]: level=INFO msg="failed to upload metrics: failed to send metrics to https://api-buzz.mon.ntppool.dev/v1/metrics: 404 Not Found (body: default backend - 404)"
Dec 19 14:02:36 debian ntppool-agent[25478]: level=INFO msg="traces export: failed to send to https://api-buzz.mon.ntppool.dev/v1/traces: 404 Not Found (body: default backend - 404)"
Dec 19 14:02:42 debian ntppool-agent[25478]: level=INFO msg="traces export: failed to send to https://api-buzz.mon.ntppool.dev/v1/traces: 404 Not Found (body: default backend - 404)"
Dec 19 14:02:44 debian ntppool-agent[25478]: level=INFO msg="failed to send logs to https://api-buzz.mon.ntppool.dev/v1/logs: 404 Not Found (body: default backend - 404)"
Dec 19 14:02:51 debian ntppool-agent[25478]: level=INFO msg="failed to upload metrics: failed to send metrics to https://api-buzz.mon.ntppool.dev/v1/metrics: 404 Not Found (body: default backend - 404)"
Dec 19 14:02:54 debian ntppool-agent[25478]: level=INFO msg="failed to send logs to https://api-buzz.mon.ntppool.dev/v1/logs: 404 Not Found (body: default backend - 404)"
Dec 19 14:02:57 debian ntppool-agent[25478]: level=INFO msg="traces export: failed to send to https://api-buzz.mon.ntppool.dev/v1/traces: 404 Not Found (body: default backend - 404)"

Could this have to do with the recent monitor software upgrade? The ntppool-agent was already spamming the system log as it is, as it is sending all messages with the same log level to the system log, despite its internal log level differentiation, but this is adding even more spam.

Would it be possible to somehow address this so that the system log isn’t spammed anymore, please?

(On relevant nodes, where I often need the system log for other purposes, I typically disable the logging of ntppool-agent completely. But I often also found that after something happening, I would have liked to be able to look at the logs. So actually getting this fixed would be preferable over simply quieting ntppool-agent completely.)

Also, I note that api-buzz.mon.ntppool.dev only has an IPv4 address. However, recently, one of my dual-stack nodes lost only the IPv4 side of its connectivity, while IPv6 kept working. Not sure whether the data uploaded to that server is critical, but at least ntppool-agent was also complaining about the server not being reachable. (I had the impression the agent also wasn’t providing its monitoring results anymore, but I now don’t remember whether I investigated enough to find a causal link between the two.)

Would it be possible to IPv6-enable this server, please?

2 Likes
2

Same here…

     Loaded: loaded (/etc/systemd/system/ntppool-agent@.service; enabled; preset: enabled)
     Active: active (running) since Thu 2025-12-11 17:53:39 CET; 1 week 0 days ago
 Invocation: 9de26582343b4b8ab842e12d91c2f3ab
   Main PID: 1280296 (ntppool-agent)
      Tasks: 12 (limit: 9342)
     Memory: 33.7M (peak: 39.2M)
        CPU: 1h 2min 51.226s
     CGroup: /system.slice/system-ntppool\x2dagent.slice/ntppool-agent@prod.service
             └─1280296 /usr/bin/ntppool-agent --env prod monitor

dec 19 17:07:55 server ntppool-agent[1280296]: level=INFO msg="failed to upload metrics: failed to send metrics to https://api-buzz.mon.ntppool.dev/v1/metrics: 404 Not F>
dec 19 17:07:59 server ntppool-agent[1280296]: level=INFO msg="traces export: failed to send to https://api-buzz.mon.ntppool.dev/v1/traces: 404 Not Found (body: default >
dec 19 17:07:59 server ntppool-agent[1280296]: level=INFO msg="failed to send logs to https://api-buzz.mon.ntppool.dev/v1/logs: 404 Not Found (body: default backend - 40>
dec 19 17:08:05 server ntppool-agent[1280296]: level=INFO msg="batch processing" env=prod ip_version=v4 monitor_ip=185.142.225.68 count=1
dec 19 17:08:05 server ntppool-agent[1280296]: level=INFO msg="traces export: failed to send to https://api-buzz.mon.ntppool.dev/v1/traces: 404 Not Found (body: default >
dec 19 17:08:08 server ntppool-agent[1280296]: level=INFO msg="traces export: failed to send to https://api-buzz.mon.ntppool.dev/v1/traces: 404 Not Found (body: default >
dec 19 17:08:09 server ntppool-agent[1280296]: level=INFO msg="failed to send logs to https://api-buzz.mon.ntppool.dev/v1/logs: 404 Not Found (body: default backend - 40>
dec 19 17:08:10 server ntppool-agent[1280296]: level=INFO msg="failed to upload metrics: failed to send metrics to https://api-buzz.mon.ntppool.dev/v1/metrics: 404 Not F>
dec 19 17:08:12 server ntppool-agent[1280296]: level=INFO msg="batch processing" env=prod ip_version=v4 monitor_ip=185.142.225.68 count=2
dec 19 17:08:14 server ntppool-agent[1280296]: level=INFO msg="traces export: failed to send to https://api-buzz.mon.ntppool.dev/v1/traces: 404 Not Found (body: default >

root@server:~# nano /etc/systemd/system/ntppool-agent@.service
root@server:~# ntppool-agent --help
Usage: ntppool-agent --env=test <command> [flags]

Monitoring daemon for the NTP Pool system

Flags:
  -h, --help                Show context-sensitive help.
      --debug               Enable debug logging
      --state-dir=STRING    Directory for storing state ($MONITOR_STATE_DIR)
  -e, --env=test            Deployment environment (prod, test, devel) ($DEPLOYMENT_MODE)
      --[no-]ipv4           IPv4 monitor (default)
      --[no-]ipv6           IPv6 monitor (default)

Commands:
  api           check API connection
    ok          Check API connection

  monitor       run monitor

  check         run a single check
    <ip> ...    IP addresses to check

  setup         initial authentication and configuration

  version       show version

Run "ntppool-agent <command> --help" for more information on a command.

Your IPv6 monitor should check him on his IPv6, if you have an IPv6 monitor running.

Yes I agree, those INFO messages may stop unless enabled, I only want servere messages like ERROR. I see no way to turn them off.

BTW, I only run IPv4 monitor only.

3

Thanks! I fixed the api-buzz.mon.ntppool.dev issue late last night (thanks to your message). Equinix Metal is shutting down, so I’ve been working on setting up new clusters. The new system will be dual stack.

3 Likes
4

Thanks @ask! Looking forward to the new clusters taking over, then, good luck with the transition!

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.