I’ve done some tests with this, and wrote my own firmware for a custom serial/USB converter. It can still talk USB Full Speed / High Speed, but provide timestamps between when the PPS happened and when the USB host picked up the carrier detect message. This was good enough for a time sync error of 7 microseconds.
Full Speed polls at a minimum interval of 1ms, which explains your results.
I have more detail and the code here: ch32v307 dev board, part 4
I’ve been meaning to design USB GPS hardware board with this firmware, but I haven’t gotten around to it yet.