Austin, Sheila
Austin contacted me about intermittent locklosses at various stages of the acquisition sequence.
He posted the attached verbal alarms log, a few interesting episodes in this log include:
P_R_M (Aug 6 23:32:38 UTC)
P_R_3 (Aug 6 23:32:38 UTC)
M_C_1 (Aug 6 23:32:38 UTC)
M_C_3 (Aug 6 23:32:38 UTC)
.....
P_R_2 (Aug 7 03:35:19 UTC)
S_R_2 (Aug 7 03:35:19 UTC)
M_C_2 (Aug 7 03:35:19 UTC)
T_M_S_X (Aug 7 03:35:19 UTC)
T_M_S_Y (Aug 7 03:35:19 UTC)
IFO_OUT (Aug 7 03:35:19 UTC)
....
P_R_M (Aug 7 04:14:22 UTC)
P_R_3 (Aug 7 04:14:22 UTC)
M_C_1 (Aug 7 04:14:22 UTC)
M_C_3 (Aug 7 04:14:22 UTC)
Verbal alarms looks at H1:FEC-(number)_ACCUM_OVERFLOW for these alarms. PR3 saturations seem suspicous because we send no ISC feedback to PR3, I looked at the osems, drive requests, and individual channel overflows and see nothing at this time, but looking at the FEC-ACCUM_OVERFLOW it does show overflows at 4:14:19 UTC. The suspensions reporting overflows at this time are all on HAM2, and all their models are on SUSH2A. Also suspicous is when there is an overflow reported from PR2, SR2, and MC2 at the same time, these are all the suspensions on SUSH34. This is what is making me think that the locking troubles may be due to some intermittent problem with CDS.
Sheila, Dave, Austin, Rahul
Following up on SUS saturation issue which Austin and Sheila faced on Sunday, I trended the ADC0 and ADC1 channels on H1SUSPRM and SUSH34.
For H1SUSPRM, I found two saturations (PRM_M3_WD_OSEMAC_BANDIM_UR_INMON), first at 23.32 UTC and second at 04.14 UTC - see attached plot. I trended all the inmons and DAQ output for PRM and did not find any of the suspension channels saturating for those two times. I am attaching a screenshot of DAQ output channels for M1, M2 and M3 stage for both the time, i.e . 23.32 UTC and 04.14UTC.
Similarly for SUSH34 also showed some saturations at channel no 24 (which is MC2_M3_WD_OSEMAC_BANDIMUL_INMON) - see two plots attached - shows all the channels in ADC0 and this plot focuses on channel 24. For MC2 I see saturations in the DAQ output for M2 and M3 stage at times coincident with SUSH34. I will further investigate on MC2 (however Sheila mentioned that it is fairly common for MC2 DAQ to saturate during locking).
Since they are also on SUSH34, hence I trended PR2, SR2 and did not find any saturations in the inmons or DAQ output.
