2nd loop QPD output alignment signals didn't make sense (attached top), and this turns out to be whitening mismatch.
This QPD goes through the QPD interface and standard whitening, and there is one unswitchable zp=0.4,40 analog whitening. This is supposed to be compensated by zp=(40,0.4) anti whitener (which is unfortunately named 0.4:40), which was off.
I turned them ON (attached left) and now it makes sense (bottom).