On Robert's request, I've created an MEDM which shows a summary of the PEM filter modules which are driving the GDS 18bit DAC outputs. The MEDM shows all three locations (4 DAC channels at each end station, 7 in the corner).

For this application, the state of; no filter control modifications and zero output counts is nominal. Any deviation from this state is flagged RED.

The screen has been linked to the SITEMAP as the last entry in the PEM pulldown.

TITLE: 03/22 EVE Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Jim
CURRENT ENVIRONMENT:
    Wind: 12mph Gusts, 8mph 5min avg
    Primary useism: 0.06 μm/s
    Secondary useism: 0.38 μm/s
QUICK SUMMARY:

Commissioners were working on getting H1 to NOMINAL_LOW_NOISE when I arrived (we are now at NLN).

Evening's Menu:

• DAQ restart tonight for TCS change. (Dave & Danny)
• PEM Injections (have heard rumblings of this work since H1 is now back to NLN)
• Commissioners are looking at H1.
• Will attempt OBSERVING when opportunities arise.

Jamie, Sheila, Dave, Jonathan,

ISC_LOCK was not connecting to H1:ASC-POP_X_RF_Q3_GAIN. From h1guardian1 was was able to connect to this channel with no problems when running caget manually in a login shell. Sheila forced a reconnection of this node's EPICS channels and was able to proceed past this point.

We noticed that htop on h1guardian1 was looking very different to restart times. After a reboot all 40 cores were evenly loaded at about 50%. Now many are in the 70-90% range, and this shifts constantly over the cores.

Jonathan installed his system diagnostics EPICS IOC on the guardian machine this afternoon. We will monitor this via StripTool over the next few days, and add these channels to the DAQ next Tuesday. I would also like to reboot h1guardian1 next tuesday so we can monitor the core usage evolution.

TITLE: 03/22 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Corey
SHIFT SUMMARY: Problems with ETMX transition, seems fixed
LOG:
15:00 Richard at EX changing ESD ground

18:30 Chris forklifting snowblower to carpenter shop

20:45 Robert to LVEA, Dave to CER

I (along with active input from Jenne) have been working on updating the medm for the violin mode monitor, which is now fully functional. You can access it from the sitemap - SUS - VIOLINS. The attached screenshot  shows the new layout. Given below are the list of improvements/additions over the old monitor.  

1. The new medm now accommodates the list of 1st (fundamental) and 2nd harmonics for the violin modes, in total 16 modes plus few unknowns, along with damping filters (BANK1 and BANK2) and monitor filters.

2. The mode frequency values have been updated, at least for the ones which has been correctly identified. There are currently 3 unidentified modes for the 1st harmonic. The unidentified ones are marked in red font (in the medm).

3. The new bar chart represents the excitation values, which is colour sensitive (green for low excitation and red for high). One can also find the mode no, freq, peak excitation values, which is listed on the window.

4. ndscopes - Now available for 1st and 2nd harmonics (including Drives, Monitors, ETMs or ITMs). The ndscopes have also been cleaned up (or refreshed) - with some old (ghost) ones removed.

After some pleading and finagling, I've got H1 locked again.  Attached is a screenshot with several DTTs, showing where this 4.2 Hz peak is appearing.  You can see that the ratio of the peak and the reference is largest in the SOFT loops, so perhaps it is getting coupled into there?  In the individual quad screen, you can see that EX has this peak much stronger than other quads, so as Sheila and Anamaria have proposed, it's likely a problem in DARM, but that is then showing up in the SOFT and then other angular DoFs. 

To get the IFO to lock, I asked Jim to wait for a long time at Power_30W for ADS to converge to the level of the noise floor, and then we also waited after Transition_from_ETMX before transitioning back to lownoise EX for the ADS to again converge to the lower noise floor. 

There were some funny changes in the Transition_from_ETMX state that I think were doomed to not work, so I've reverted all of them to how it was yesterday afternoon.  I suspect that these were why we lost lock in this state last lock.

Also, just in case the 4.2 Hz is coming from the DARM loop as a result of imperfectly matched analog and simulated L2 coil driver electronics, we've switched back to using the lowest noise L2 coil driver state.  So, I can't say if the reason I was able to lock is that I switched to the old coil driver state, or if it's because I have been waiting really, really long times for ADS to converge, but at least the combo of the 2 is working (with N=1 at least...).

Also of note, we have higher microseism today than we have been having lately, which is my suspicion for why we have worse noise at low frequency right now, which is showing up as a ~5+ hit to our BNS range.

PEM crew noticed offsets on the DAC channels they were using.  This turned out to be offsets in the filter bank for each channel.  It has been this way since they were installed.  Dave B will create an MEDM screen to make this easier to see.

Took advantage of the IFO down time this morning to restore the EX grounding to final state until better isolation scheme is implemented.   The ESD is only grounded at the beam tube.  Power supplies lifted at the rack.  All other supplies grounded at the rack.

TITLE: 03/22 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Ed
CURRENT ENVIRONMENT:
    Wind: 3mph Gusts, 2mph 5min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.29 μm/s 
QUICK SUMMARY:
Stuffs broken.

TITLE: 03/22 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Jim
SHIFT SUMMARY:
LOG:

8:44 NLN

9:01 Intention Bit Observing

12:33 Lockloss

12:34 Wrestled the Y arm back into submission

12:45 Begin full locking sequence

13:01 no go. Arms need too much tweaking. Georgia suggested IA upon lockloss. After 1 attempt without, I'm giving in.

13:02 Begin Initial alignment

13:33 Begin full locking sequence

14:01 Lock failed at transition to ETMX - LSC Split Servo Board Voltage is High

14:15 Bubba out to EY chiller yard

14:27 Lock failed at transition to ETMX

14:30 Bubba back

14:31 Going to hold at LOWNOISE_COIL_DRIVERS until commissioners get here

14:40 WELL....THAT DIDN'T WORK

14:43 Richard out to EX - lift ground on ESD at power supply rack

14:55 Leaving IFO at CHECK_AS_SHUTTERS for commissioners

12:34UTC

H1 Locked at NLN/Observing (91.8Mpc) for 2.25 hrs. First big EX glitch as Im writing this. Possible signs of ~4Hz oscillations occurring. ASC_DHARD_YAW_+OUT16 used as a reference. Also, witnessed in yaw and CHARD traces.

To get to observing tonight we accepted a SDF diff that I don't understand:

The gain of ASC-POP_X_RF_Q3 went from 3.95 to 2.8. All the other quadrants have a gain of 3.95.

Anamaria, Georgia

This evening we struggled to get past LOWNOISE_ESD_ETMX several times. The ~4.2 Hz instability kept ringing up and killing the lock almost instantly. We see this instability in ETMX L2 drive, the ASC arm degrees of freedom, and in DARM. We're not sure why this problem suddenly became so egregious. Earlier today we changed the coil drivers state for L2, maybe this is related?

(Recall: in the TRANSITION_FROM_ETMX state we transition DARM control from ETMX L1/L3 to ETMY L1/L2 and ITMX L3. In LOWNOISE_ESD_ETMX we transition all 3 stages back to ETMX. )

Things we tried:

• We tried to transition only L1 and L2 back to ETMX, leaving L3 on ITMX, but we lost lock due to the 4.2 Hz instability in L2, so the ESD is not the problem.
• We tried to take some PUM/UIM crossover measurements, but had problems getting sensible coherence.
• We tried reverting the changes made at the end of February (a gain increase and a boost at 4.2 Hz in ETMX_L2_LOCK filters), and saw the same sort of lockloss we had back then (L2 and L3 ringing up at the same frequency but out of phase).
• We tried only lowering the ETMX_L2_LOCK gain, while leaving the boost in. This seems to have done the trick.

We lowered the ETMX_L2_LOCK gain to 17 (from the nominal value of 23). Anamaria noted that the 4.2 Hz peak was still visible in DARM and occasionally dominated the RMS, so we lowered it again to 15. I put this in guardian and loaded it.

I updated the front end calibration actuator filter (CAL-CS_DARM_FE_ETMX_L2_LOCK_GAIN) to match the new actual gain.

9:01UTC Intention bit set to Observe

9:18 Georgia reloaded the ISC_LOCK guardian which knocked the bit back to Commissioning

9:18 Intention bit reset to  Observe

~ 07:17UTC I noticed the Hardware Watchdog indicators sort of cycled RED , momentarily, only on the left half of the otherwise GREEN indicators of the CDS Overview screen right after POWER_30W. Peculiar?

TITLE: 03/22 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Travis
CURRENT ENVIRONMENT:
    Wind: 13mph Gusts, 10mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.19 μm/s
QUICK SUMMARY:

Holding locking sequence (if we get there) at LOWNOISE_COIL_DRIVERS while Anamaria and Georgia hash out the ongoing issues.