After consultation with Bubba and Cheryl, I'm silencing this cell phone alarm while Bubba is investigating the issue.
Bypass will expire:
Fri Aug 9 08:43:32 PDT 2019
For channel(s):
H0:FMC-EY_CY_H2O_SUP_DEGF
I tested going to observing without squeezing during the commissioning break today. The previous instructions did not work because the SQZ_MANAGER's subordinate nodes also needed their nominal states changed, and this is not obvious on the node screens. I talked to Jamie about this and filed an issue for it.
The new instructions have been updated in this wiki. There's a quick reference up top, and overly detailed instructions below. The basic idea is:
Request SQZ_MANGER to NO_SQUEEZING
SQZ_MANAGER -> NO_SQUEEZING
SQZ_OPO_LR -> DOWN
SQZ_CLF_LR -> DOWN
SQZ_LO_LR -> DOWN
Manual ISC_LOCK to INJECT_SQUEEZING, AUTO then back to NOMINAL_LOW_NOISE
TITLE: 08/08 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 111Mpc
OUTGOING OPERATOR: Ed
CURRENT ENVIRONMENT:
Wind: 20mph Gusts, 15mph 5min avg
Primary useism: 0.09 μm/s
Secondary useism: 0.07 μm/s
QUICK SUMMARY: in Observe, thunder storms expected and currently heavy clouds and some rain
Occasionally the transition to the earthquake mode has caused locklosses. It took me a little while to figure out what was causing it, but I think it's mostly due to the BRSs adding a lot of very low frequency signal to the sensor correction signal, plus insuffficient low frequency roll-off of the sensor correction filter we use just for the earthquake mode. I've made a different filter (first attached plot) that has more low frequency roll-off, but gets slightly worse subtraction around 50-80 mhz. The dotted blue line is the new filter, the red is the old Eq filter and the gold is the filter we use during non-eq times. The new filter still injects more low frequency motion into the ISI cps below 10 mhz than the nominal filter, but less than the old filter. The reason we want such a low frequency filter is so we can try to suppress differential motion in the earthquake band, which the nominal filter only amplifies.
The second plot is a comparison of the outputs of different sensor correction filters during a earthquake when the transition broke the lock. The purple trace is POP A, showing when we lost lock, the dashed green is the seismic state, we lost lock 14 seconds after the request was made by the operator. The other 3 traces are the outputs of the different sensor correction paths for the Y arm, which are different ground signals run through different filters. The blue trace is the eq path with the new eq filter, the red trace is the old eq filter. The gold trace is the nominal sensor correction filter and path. I don't post it here but the low frequency motion (< 10mhz) was much smaller on the X arm at the time.
The reason we lost lock was the DARM offset created by ramping from the gold trace to the red trace, pushing all of the chambers ~10 microns in the y direction over 30 seconds. The new filter doesn't create such a large offset, hopefully this will lead to fewer lock losses transitioning to the earthquake state.
I'm attaching asds of the STS SCSUM signals for the different filter configurations. The blue trace is the old eq filter using the local differential ground. The red trace is the new eq filter using the local differential ground. The gold trace is the nominal sensor correction filter using the local total ground. The peak displacement for the new filter is about a third of the old filter, which more or less agrees with the timeseries I posted above.
TITLE: 08/08 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 113Mpc
INCOMING OPERATOR: Cheryl
SHIFT SUMMARY:
Some commissioning issues with the squeezer laser power and there was a Superevent. See previous aLogs.
LOG:
15:14 Initial Alignment
15:39 Re-Locking attempt
16:27 NLN
16:32 H1 Observing
17:15 Joe B called to inform me that one of our squeezer temps was railing
17:24 H1 out of Observing - Squeezer unlocked - I'm flagging this as corrective maintenance as Daniel and Keita are on it.
17:54 DS and co, out to LVEA to remedy low squeezer laser power.
21:01 H1 into Commissioning mode to address squeezer issues and PRCL/SRCL FF issues
22:27 NLN
22:27 H1 back to Observing
22:27 Superevent S190808 aka G346647
23:00 Handing off to Cheryl
Confirmed with LLO
INJ_TRANS to INJECT_KILL
Stand Down in effect until released by Keita
[Kara, Jenne, Sheila]
We took a suite of core noise budget injection measurements, to check our status with the new alignment we've been using the last week or so. Note that we have taken measurements for the LSC FF to be updated, but have not done the update yet. Note also that these were done with the squeezer off (usually we do them with the squeezer on). The injections are large enough that that shoulndn't significantly impact our ability to estimate the noise TFs.
We took injections for:
We also had some quiet time with no squeezing.
[Kara, Jenne]
We took MICH, PRCL, and SRCL LSC FF measurements today since we see that we have increased coherence with SRCL with the new alignment we've been using for the last week. MICH doesn't see increased coherence, but we took its data for completeness. We also hope that this PRCL measurement will go to low enough frequency to make the fitting easier, so we can implement PRCL FF for the first time.
21:01UTC to address squeezer issues and PRCL/SRCL FF issues
Back to Observe, with one SDF accept for squeezer nominal setting in a settings checker.
I just realized that in the scuffle of fixing the squeezer, clearing diffs and getting back to Observing while Verbal was shouting Gamma Ray Burst at me and trying to see if the parameters were such that I needed to stand down the IFO, that I forgot to put INJ_TRANS into INJECT_KILL. smh
D Sigg S Dwyer Daniel found that the squeezer laser was multi mode after scanning the SHG from the control room. squeezer laser current now: 1.904A when we got here: 1.870A Things seem to be back to normal for now.
Laser power started droppping slowly about 2 days ago. It went multimode a little less than 3 hours ago.
Didn't last. Another current increase to 1.950A. The output power of the squeezer laser has degraded by 10% over the last 2 days. This may be an indication that the pump diode is dying.
The first attachment shows about 5 months of trends of the squeezer laser powers both before and after the SHG, the right top panel shows power out of the laser in yellow and the power out of the SHG in green. The earlier episodes of mode hopping show up as decrease green out of the SHG (because the IR power is in the wrong mode) but the IR power doesn't drop. The last month and a half have been the most mode hop free we've had from the squeezer laser. The last couple of days are the downturn at the end of the plot, showing that this incident is different in that the power out of the laser is dropping, quickly, as Daniel notes.
Fermi alert - Trig Duration .512
Standown 1 hr
Cancel that - Squeezer unlocked again -_-
18:14UTC Stand Down is ON AGAIN.
Around 7am, Daniel came in to check on H1. He was seeing the same issue I had, and I heard him mentioning how the y-arm was locking on the wrong modes & he couldn't tweak optic around to remedy this. So he was looking into locking to see what Guardian might be doing.
Eventually he finally got the 0:0 mode back! And it turns out it was a gain change.
He mentioned that the Y-arm thought it was locking the right mode, but in reality it was not. So a lower order mode was continually being locked (& guardian might have even been trying to enable WFS on this bad mode---which is not good!). Looking over Daniel's shoulder I saw that he had the PDH Locking medm open and here he was looking at the Acquire Gain.
With this high gain, Y-arm was thinking a bad non-0:0 mode was what we want, but we had an ugly 1:0 mode trying to lock. 
So, Daniel lowered the Acquire gain so we would no longer be fighting autolocking.
Then he unlocked the Y-arm and was now not fight the autolocker which was trying to lock a 1:0 mode. So, now he could move in yaw (I belive he said ETMy about 1.0 counts) beyond this 1:0 mode and was then able to get the Y-arm back into 0:0 territory/good alignment. And just like that Y-arm was back.
Attached is a screenshot of the PDH Autolocker for the Y-arm and you can see the Acquire Gain on the lower right.
(Daniel is adding a more thorough alog now.)
The Y arm has had bad mode matching since our last big vent, which often means that the automation won't work unless the arm is well aligned. Jason has been working on the mode matching problems for the last several Tuesdays. Normally the situation can be handled by taking the Y arm guardian to unlocked, watching the flashes and manually adjusting the alignment until the flashes reach the normalized value of 1. Then returning the guardian to the state you found it in will usually allow you to keep moving. Was this situation different?
After the lockloss, I did the following:
Daniel noticed the same occurrence of the TEM 01 flashing being more prone to lock and with higher powers than the TEM 00 flashes.
So this led Daniel to changing the Acquire gain. Then he made yaw adjustments to get us out of the TEM01 yaw misaligment state and (fairly quickly) he was able to get to an alignment where the TEM 00 would lock at powers approaching 1.0 (vs TEM 00 locks which I tried had values of 0.2 or less).
This was definitely different my usual experience of tweaking a green arm back to a good alignment. Not realy sure what made this situation different than the normal y-arm tweaking we've been used to for the last few months.
I wonder if running the baffle align scripts would have been helpful in this case. Hopefully they would have helped put the optics in a slightly better place for catching a good mode.
Oooo. Yeah, that would have been something worth trying, but I actually have not run them (I recently looked over Jeff K's shoulder as he ran them on a recent Maintenance). How does one run a baffle alignment?
TITLE: 08/07 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Cheryl
SHIFT SUMMARY:
LOG:
16:11 switching SEI_CONF mode back to WINDY - noticed it had been in EQ mode for ~11 hours.
17:38 Vanessa to MX
19:43 TCSX went down
19:50 Lockloss
20:00 begin 2 unsuccessfull re-lock attempts
20:37 Initial alignment
20:44 Hugh and Jim heading out to floor to make measurements in H2 diagonal.
20:59 Hugh and Jim back
21:04 begin re-locking
22:56 Verbal Alarms seems to have lost history again -
22:58 Handing off to Cheryl
I'm leaving this here as a reminder, the VerbalAlarms wiki shows where to find the rercord of the Verbal logs. It's here:
/ligo/logs/VerbalAlarms/Verbal_logs/{year}/{month_number}/
There is a txt file for each day (in UTC times) that you can bring up in a text editor or use "cat filename" to view in the terminal.
called and talked to Bubba, both pumps are on at EY, the VEA temperature is holding steady, plots attached show 7 days and 1 day of EY and EX.