@ask, not specific to chronyd, but here another occurrence of “graph shrinkage”:
In this case, the shrinkage seems caused by trying to graph offset values that can be assumed to be invalid/nonsense based on the INIT kiss code and the leap indicator value of 3. The CSV log does not show the stratum, would be interesting to see what value it had in this case (I would expect it to have been 0).
Apart from the effect on the graph, the INIT kiss code as well as probably at least the similar STEP kiss code are errors that come to mind with respect to the question how to score “errors […] that have a response but aren’t RATE”, RSTR, or DENY. Others, e.g., related to failures in security mechanisms the polled server expects, could be worthwhile to consider as well.
EDIT: I faintly remembered this had been raised before, but initially did not find it. But here it is.