offloading green WFS in each lock

Yesterday Jenne changed the normal path we take for the ALS ARM guardians so that green WFS are run to both TMS and the ETM in the normal locking sequence.  

Today while Jenne and Georgia are working on the ISS 2nd loop AC coupling, I made some states that can offload the alinment from the WFS to the sliders after the green WFS run.  I've tested it a few times for both arms, and it seems to work well, so I've changed the request in LOCKING_ARMS_GREEN to offload this alignment.  This will make LOCKING_ARM_GREEN take a bit longer, but it should help us to be better able to relock the arms after a lockloss, since we will keep the green alignment that worked in the previous acquisition sequence.  

This can be reverted by changing lines 628 and 629 in ISC_LOCK to:

        nodes['ALS_XARM']  = 'LOCKED_SLOW_WFS_ETM_TMS' 
        nodes['ALS_YARM']  = 'LOCKED_SLOW_WFS_ETM_TMS'

Note:  I've reverted this, because the guardian was moving on too quickly and trying to offload before the WFS had really run.  We added a convergence check, but haven't put the offloading back in the normal path yet.

BRS Trends

Despite an adjustment on Tuesday, neither of the BRS's look like they are centered that much better. However, both are safely within the warning thresholds.

Ops Day Shift Transition

Ops Shift Transition: 05/31/2019, Day Shift 15:00 – 23:00 (08:00-16:00) - UTC (PT)

State of H1: Locked

Intent Bit: Observing

Weather: Low wind, clear sky

Primary 0.03 – 0.1Hz: 0.01 um/s

Secondary 0.1 – 0.3Hz: 0.1 um/s

Outgoing Operator: Travis

Quick Summary: Locked and Observing for 5.5 hours, through some strong EQ's. Not much seismic activity/wind currently

Ops Owl Shift Summary

TITLE: 05/31 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 111Mpc
INCOMING OPERATOR: Niko
SHIFT SUMMARY:  No issues to report after relocking.  Rode through several EQs without issue.
LOG:

7:29 GRB alert but we were unlocked at the time

7:50 Started initial alignment.  Relocking seemed to be going well after the lockloss, getting past DRMI a couple of times, then things started to walk off.

9:41 Back to Observing

10:23 SEI_CONF taken to EARTHQUAKE in response to 6.1 Mag EQ in Palau

11:57 5.8 Mag EQ in Mexico.  We were still in EARTHQUAKE mode for SEI, so we survived.  LLO did not.

12:02 GRB alert

13:01 GRB alert

13:38 GRB alert

14:00 SEI_CONF returned to WINDY

Tagging @DetChar and SEI for data in seismic's sensor correction configuration transition studies.

Observing at 9:41 UTC

Relocking took longer than usual, mostly waiting for various stages of ASC and SOFT loops to converge (some of them were WAY out).  Screenshots of SDF diffs accepted attached.

Ops Owl Shift Transition

TITLE: 05/31 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Patrick
CURRENT ENVIRONMENT:
    Wind: 9mph Gusts, 7mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.11 μm/s
QUICK SUMMARY:

Lockloss 1 minute before my shift actually started.  Cause unknown. 

 

Ops Eve Shift Summary

TITLE: 05/30 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Travis
SHIFT SUMMARY: Lock loss at very end of shift. Cause not apparent. Briefly taken out of observing by squeezer near end of shift. SQZ_MANAGER then had user message: 'green power stabilization railed'. Log had it printed at 06:15 UTC. Dust alarm in vacuum prep lab near start of shift. 
LOG:

22:54 UTC Kyle back from mid Y
06:14 UTC Knocked out of observing by squeezer
06:16 UTC Squeezer automatically relocked. Back to observing.
06:58 UTC Lock loss

Ops Eve Mid Shift Status

Have remained locked and in observing. No issues.

Why can't we AC couple the ISS in lock?

Keita, Georgia, Jenne, Sheila

We spent a bit of time today investigating the lockloss which occurred when we tried to re-AC couple the ISS second loop. During lock acquisition the second loop is engaged with its AC coupling servo on. Then in LASER_NOISE_SUPPRESSION we DC couple it, by holding the output of the AC coupling servo. In order to increase the input power to the IFO once we've already reached nominal low noise, we need to be able to re-engage the AC coupling servo, which it turns out is very fiddley.

In December last year Stefan cleaned up a guardian state to do this, it worked then when he tried it with the IMC. What this state actually does is:

• servo the offset of the reference_servo such that the output is zero
• servo the offset of the AC_coupling_int (integrator) such that the output is the same as the previously-held output (i.e. the value that we held the output to be when we DC coupled the ISS)
• re-engaging the AC_coupling_drive, but with a 1Hz high-pass filter in the AC_coupling servo (FM6), which essentially AC-couples the AC-couping servo, i.e. leave the ISS DC coupled
• Turning off FM6 (note it has a ramp time of 15 seconds), so the ISS is now fully AC coupled

Checking out the guardian log (first attachment), today we survived the first 3 steps, but lost lock straight after the 4th step of turning off FM6 to actually AC couple the loop. Keita did some sleuthing and found out why. There appears to be an offset on the input to the AC_COUPLING_SERVO (brown trace in second attachment). When we turn off FM6 (blue trace shows the switch stat), the output of this servo (orange) is changes to make the input zero. Unfortunately this drive to the AC couple servo causes a large change in the output of the AC coupling drive (red), and coupled to the ISS first loop, driving the IFO input power down.

Tomorrow we want to step through the guardian state by hand and manually add offsets (eg to the SECONDLOOP_EXCITATION_OFFSET channel) to bring the AC_COUPLING_SERVO input closer to zero before turning off FM6. If that works by hand we can add another adding-offsets-to-drive-something-to-zero type servo to the guardian.

Shift Summary - Day

TITLE: 05/30 Day Shift 15:00 – 23:00 (08:00-16:00), all times posted in UTC

STATE of H1: Observing

INCOMING OPERATOR: Patrick

SHIFT SUMMARY: Lost lock around 16:00 UTC trying to get to a state where we can raise the power to 40W. Relocked and went to 40W but saw an ADS line starting to drift away from 0 and went back to 35W. Transitioned to Observing, and have been so for 4 hours.

LOG:

15:00 (08:00) Start of shift

15:29 (08:29) Richard, contractors to MX, maybe outside EX

16:09 (09:09) Lockloss trying to get to a state where we can raise power to 40W

16:44 (09:44) Pep to E-bay -- move magnetometers

16:51 (09:51) Pep back from E-bay

17:02 (10:02) Sheila to LVEA -- check frequency plugin

17:12 (10:12) Kyle to MY -- measuring hole locations

17:31 (10:31) Vanessa to MX

18:31 (11:31) Kyle back from MY

19:14 (12:14) Going into Observing

20:37 (13:37) Kyle to MY -- continue work

21:28 (14:28) Jason and Ed to MY

21:52 (14:52) Jason and Ed back from MY

23:00 (15:00) End of shift

Ops Eve Shift Transition

TITLE: 05/30 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 109Mpc
OUTGOING OPERATOR: Niko
CURRENT ENVIRONMENT:
    Wind: 16mph Gusts, 13mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.09 μm/s 
QUICK SUMMARY:

No immediate issues.

H1 down time due to wind

Laurence Datrier, Sheila Dwyer

 

We looked at O2 data for H1 to get an estimate of down time due to wind. We used the maximum wind speed of the three max. 1min and 30min trends for the H1 EX, EY, CS wind channels, for all of O2.

The first histograms and trend line show duty cycle vs wind speed percentiles, for wind speed vs lock status and previous wind speed (max. of previous 1 or 30 min) vs lock status. Looking at the previous wind speed shows more down time due to high winds. Percentiles and wind speed values are (in mph):

percentile	5	10	20	30	40	50	60	70	80	90	95	99
1min trend wind speed [mph]	3	4	5	6	7	9	11	13	16	21	25	33
30min trend wind speed [mph]	5	6	7	8	10	12	14	17	21	26	31	41

 

The following histograms are the normalised distributions for wind speed at:

•  Instant of lock loss (1min and 30min trend)
•  Instant before lock loss (1min and 30min trend)
•  Time leading up to lock loss (1min trend: 5mins up to lock loss, 30min trend:2.5hrs up to lock loss)

Compared to the normalised distribution at all times (blue). A significant tail appears in the distribution for wind speed at time of lock loss for winds >~35mph.

Accept SDF Differences after Relocking

   Accepted the following SDF differences upon relocking after the El Salvador earthquake.