Taking the opportunity while both sites were down this afternoon, EvanG and I went to EY to investigate the possible cause of clipping at the PCalY RX PD as noted in JeffK's aLog 33108. Looking at the beams in the RX enclosure, Evan and I agreed that the clipping appeared to be coming from the newly installed alignment irises which were installed the last time we fixed a clipping issue at the same end station. We also agreed that the least invasive fix to the issue was to move the iris, rather than move the steering mirrors in the TX module as had been done the previous time. We still don't understand why these beams are moving (perhaps the mirrors on the in-vac periscope are moving, as has been hypothesized?, or the steering mirrors in the TX module are mysteriously drifting), but both inner and outer beams were entering the input aperture of the RX PD integrating sphere after moving the iris, so we decided to stop there.
In attached ASD, the GREEN trace is the current measurement after fixing the clipping.
The 13 day trend of the RX PD also shows that it has returned to pre-clipping values.
Evan G., Travis S., Jeff K. Looking back at the calibrated Pcal TX and RX PD trends over the last 90 days, it appears that a small amount of clipping might have began shortly after the iris apertures were installed near the end of October (see first attachment and LHO aLOG 30877). Also observed from this trend, the EPICS records are updated on Nov 7 (see LHO aLOG 31295); a slow trend continues until about mid-December when variations becoming more apparent; shortly after Jan 4, the clipping becomes more severe with much larger excursions; and finally, Travis and I fixed the clipping, returning the trend back to the nominal value. Jeff and I were concerned this might impact the reference model calibration measurements made early Jan 4 UTC. Looking carefully at the trend, Jeff's measurements happen at a very fortuitous time. The excursion from the nominal, good Pcal state are extremely small, so the impact on the measurements made for the O2b reference model are negligible. To investigate the cause of the variations over the last 2 weeks trended the temperature of the VEA to look for correlations between temperature and the observed fluctuations (see second attachment) over the last 13 days. The larger temperature changes from Jan 3-Jan 4 correlates with the measured light at the RX PD and changes at the 1% level. The large, rapid temperature change on Jan 6 impacts the light measured at the RX PD by 13%. Then, following this, the temperature holds steady while we observe variations at the few-percent level, indicating the alignment was brought into a bad state. These variations over the last two weeks means that the time-dependent calibration factors (computed from the RX PD signal) are impacted at the few-percent level, larger than the requirement. We might need to fall back to using the TX PD as the reference for time-dependent calibration factors. The estimated uncertainty of the photon calibrator (without clipping) is 0.75% (see P1500249). Also note from the first attachment that the ratio of TX and RX PD channels is ~0.2% (without clipping), well within the uncertainty of the Pcal. We wondered if these variations might impact the range, but the variation and trend of the range do not appear to correlate by eye with the variations in the RX PD trend. In summary, the reference measurements are not impacted, but the time-dependent calibration factors are impacted at the few-percent level until the clipping problem was fixed.