TITLE: 05/05 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 157 Mpc
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY:
IFO is in NLN and OBSERVING as of 05:57 UTC (1hr 17 min lock)
Much of the shift was spent troubleshooting locking issues in the interferometer. There were mainly two issues. (lockloss alog 77620 and alog 77615).
BS Glitch/PRCL Buzzing
This issue was characterized by locklosses that occured during TURN_ON_BS_STAGE_2 and ENGAGE_DRMI_ASC. There would be a kick seen primarily in the ST1 T240 and L4C channels (Screenshots 1 and 2). Initially, this was thought to be a BS glitch. I first noted this issue in alog 77615 but as this alog states, we have been experiencing this for quite a while. During the DAY shift, Sheila and Tony found that in fact, this may be an issue with the PRCL gain being too high, causing a 100Hz buzzing (Screenshot 3). Relevant alogs include: alog 77574, alog 77573 (which detail Solution 1), and alog 77583 (Solution 2)
Temporary Solution 1:
Wait in ENGAGE_DRMI_ASC for ASC to converge more than guardian needs it to in order to mitigate the effects of possible BS kicks. From roughly 9-10 locklosses of this species, this worked around 6-7 times.
Temporary Solution 2:
Request and stay in ACQUIRE_DRMI_1F, change the PRCL1 gain from 8 to 4, continue to TRANSITION_DRMI_TO_3F and then change it back. This worked every time I tried this (3 times) and is the better solution.
Transition from ETMX
This issue was characterized by locklosses that occured either towards the very end of TRANSITION_FROM_ETMX or just as guardian reached LOWNOISE_ESD_ETMX and is part of an ongoing trend that is mostly detailed in alog 77359 where every time this state is run, there is a glitch in the 10-15 seconds leading upto the transtion of states. The alog notes that while investigations are ongoing, this seems to be something that ramp time and filter changes would fix. (screenshot 4 and 5). The alog also notes that locklosses in this state are most prevalent during periods of high wind. This issue has NOT been fixed and we were only able to survive it due to decreasing wind microseism. As we approached this state, after losing lock here twice, I checked the wiki/Jenne's troubleshooting doc and made calls to 3 people on the call list but either they could not help or there was no answer. Thankfully, a 4th person's Saturday night wasn't interrupted because the wind calmed down enough to survive this. Screenshot 4 shows a lockloss during this state while Screenshot 5 shows us surviving it - both have the glitch.
Other:
- Initial alignment ran fully automatically, which the lock acquisition survey shows did not happen recently. Initial alignment did not fix either issue.
- Find_IR was particularly slow, failing once and requiring DIFF intervention.
- ALS was particularly unstable after the second lockloss.
- DRMI was actually catching quite quickly (but not staying through to BS_2 due to aforementioned issue) - in 10 DRMI locks, we only went through PRMI twice and we did not go through MICH at all.
- Violins looked splendid the whole time (Thanks Rahul and Tony!)
- We survived a 5.6 Mag EQ
I expect to continue investigating both these issues tomorrow and wanted to get to them today but spent around 6.5 hours locking and 0.5 hrs eating. Given we are in observing for sufficiently long, I will look into each lockloss per Sheila's on the phone and alogged advice (and hopefully find with something meaningful to alog).
LOG:
None
