Locked in Nominal Low Noise and Observing. Minimal glitches, no saturations so far, and the environment seems calm.

At 19:14 PST there was a critical alarm on H0:FMC-EX_CY_H2O_SUP_DEGF. By the time I got to it it was no longer in alarm. The EX chiller medm seemed to look normal to me, but trending the previously mentioned channel did show an odd flat spot in the supply temperature for the past ~3 hours (See attachments below). The tempurature began to increase again at 19:07 and seems to have gone back into its normal rythme.

First screenshot: A 12hour trend from today

Second screenshot: A 12hour trend from yesterday

I will continue to montior this.

Title: 9/19 Day Shift 15:00-23:00 UTC (8:00-16:00 PST).  All times in UTC.

State of H1: Observing

Shift Summary:  After a few false starts at lock stretches of significant length, H1 has been locked in Observing Mode for the last ~5 hours of my shift.  A few glitches related to ETMy saturations, but otherwise stable.  Environment: calm.

Incoming operator: TJ

Activity log:

18:43 Evan arrives on site

DCS copies the H1_R raw science frames from CDS to the LDAS archive and the H1_C commissioning frames from CDS to the LDAS scratch filesystems. Currently the commissiong frames are also being copied to the LDAS archive. To fix a load issue on the LDAS side which was causing archiving of the science frame to fall behind, this latter process has been switch to primarily get its data from fw1. LDAS is getting the science frames from fw0, but now is getting commissiong frame for both scratch and archive from fw1. This should not affect CDS.

Once again back to Observing @ 18:11 UTC.

Lost lock again @ 17:47.  ITMx saturation, but again not obvious enviromental culprits.

Locked Nominal Low Noise @ 17:40, set to Observing Mode.

H1 has been relocked and set to Observing Mode at ~76 MPc.  I had to accept 2 SDF Diffs (H1:ALS-Y_WFS_DOF_3_P_GAIN and H1:ALS-Y_WFS_DOF_3_Y_GAIN) in order to go to Observing Mode.  There is still a notification present about SPM Diffs for SEI HAM5 (H1:HPI-HAM5_MASTER_SWITCH and H1:ISI-HAM5_MASTERSWITCH), but these did not prevent me from going to Observing, so I am ignoring them. 

Also, going through the checklist before setting Observing Mode, I had a question about sweeping the LVEA.  I couldn't find an aLog about this being done, so I called Corey.  He stated that this was being postponed until next Tuesday and that I could safely ignore this for now.  Nevertheless, the lights in the LVEA and PSL are all off and no one is on site besides Elli and Miguel who are in the CR. 

Title: 9/19 Day Shift 15:00-23:00 UTC (8:00-16:00 PDT). All time posted in UTC.

State of H1: Lock acquistion.

Outgoing Operator: Jeff Bartlett

Summary: A series of EQs in Chile have had the IFO down for most of the OWL shift.  They have rung down enough now that we should be able to lock.  Alignment looks good, so I will attempt to relock without initial alignment first.

Activity Log: All Times in UTC (PT)

07:00 (00:00) Take over from TJ
07:00 (00:00) IFO relocking after EQ lock loss
07:16 (00:16) IFO locked at NOMINAL_LOW_NOISE, 22.9W, 72Mpc
07:22 (00:22) Set Intent Bit to OBSERVE 
08:05 (01:05) ETMY saturation
09:56 (02:56) ITMX saturation
09:56 (02:56) Lockloss – 6.1 Mag earthquake near Chile
10:50 (03:50) IFO not locking – Run initial alignment
12:31 (05:31) While at LOCK_DRMI_1F BS ISI stage 2 WD trip – Reset watchdog
13:30 (06:30) BS ISI stage 2 WD trip – Reset watchdog
13:34 (06:34) Set IFO to DOWN while ring down from last EQ
14:34 (07:34) Received a GRB alert. IFO is currently down
15:05 (08:05) Turn over to Travis


End of Shift Summary:

Title: 09/19/2015, Evening Shift 07:00 – 15:00 (00:00 – 08:00) All times in UTC

Support: 

Incoming Operatror: Travis

Shift Summary: 
- 09:56 (02:56) – Lockloss 6.1 mag EQ near Chile – Seismic activity rung up, but coming down rapidly. Will attempt to relock.
- 13:25 (06:25) – Another 6.2 mag EQ near Chile – Seismic activity has rung up again. Continue with relocking attempt.
- 13:34 (06:34) – IFO repeatedly coming up to LOCK_DRMI_1F and higher before losing lock. Put IFO in DOWN state until seismic activity calms down. Spoke with LLO they are also holding for things to settle down. 
- 13:45 (06:45) – Another 5.4 mag EQ near Chile 
- 14:34 (07:34) – GRB alert. IFO down at this time

Recovering from 09:56 (02:56) lockloss. Relocking failing at various points along the Guardian path. Had DRMI_1F catch in split mode a couple of times. A tweak of the BS corrected one, went down on the other before I could catch it.  

Transition Summary:
Title:  09/19/2015, Owl Shift 07:00 – 15:00 (00:00 – 08:00) All times in UTC
State of H1: Unlocked 
Outgoing Operator: TJ
Quick Summary: IFO is relocking after lock loss due to earthquake. LLO also recovering at this time. 
  

All times posted in UTC

STATE Of H1: Relocking after lockloss at 5:55, currently have DRMI locked.
SHIFT SUMMARY: Most of the shift was quiet except for a few ETMY saturations and one OMC DCPD.
INCOMING OPERATOR: Jeff B.
ACTIVITY LOG:

ETMY Saturations:

• 23:15
• 0:14
• 0:18
• 2:10
• 4:06
• 5:27
• 5:45

OMC DCPD Saturation - 2:10

Lockloss - 5:55

5:55 - Lockloss

As I was typing up a report saying how well things have been going, the ASC Striptool started looking bad. After a few minutes of this, it lost it.

I had a look at factors that impact our duty cycle, starting at midnight UTC Sept 10th until 2:43 UTC Sept 18th (the latest time available when I started downloading data.) I choose this time because there were fewer of the commisioning activities that maade it hard to get a good picture of the duty cycle in the earlier weeks of ER8.

• We were in the nominal low noise state for 68% of the week, although this includes about 10 hours of time when the data quality was very poor because of a beckhoff failure which introduced a 2Hz comb. (21530, 21555)
• We were not attempting to lock (in DOWN or READY) for 13.7% of the week.  This included earthquakes, alignment (at least 4% of the week), and time when we were down for beckhoff troubleshooting (about 9 hours or 5% of the week).
• We are still spending about 8% or our time locking DRMI. The median time to lock DRMI is still 5 minutes, and we still have a handfull of long attempts (2 and a half hours was the longest this week)
• Our lock acquistion sequence is more reliable than it was.  Out of 49 DRMI locks, 33 survived to resonance, 28 to DC readout, and 23 to nominal low noise.  This is an improvement over the situation durring ER7 when about 1 in 5 DRMI locks survived to the low noise state.
• The third plot shows a histogram of the length of our low noise locks.  Half of our locks have been less than 5 hours long, with the longest 25 hours. 

Ground motion

• Although we think that we have improved our ability to lock on windy days, the third attachment shows a plot of wind vs duty cycle which does not seem to have changed much since ER7 (see the equivalent plot for ER7 here).  The low wind speed and high wind speed points are based on small amounts of data. This week has been less windy than normal, with winds above 15 mph only 15% of the time.
• There is also a similar plot for the Z direction blrms of low frequency ground motion, which should help us understand the impact of earthquakes.  We've had many locklosses durring even small earthquakes this week and have had trouble relocking until the ground calms down.  I am not sure about the units on this channel, it is calibrated but it doesn't seem to agree with the DMT monitor on the wall.  I will update this entry when I know.  (hints?)

It seem that we've taken much of the low hanging fruit in making our locking sequence more robust, while there are still parts that could be speed easily up the gains to be had from doing that are pretty small.  Probably the most important thing we can do to improve the duty cycle now is to figure out why we are loosing lock durring relatively small earthquakes.  Another alog about that comming soon. 

TITLE: 09/18 [DAY Shift]: 15:00-23:00 UTC (08:00-16:00 PDT), all times posted in UTC
STATE Of H1: Observing, ~70 Mpc
SHIFT SUMMARY: Remaining half of shift quiet
INCOMING OPERATOR: TJ
ACTIVITY LOG:

19:36 - 19:41 UTC Stepped out of control room
20:52 UTC Turned off test points I hadn't realized I had open
21:04 UTC Large glitch in range, but did not see corresponding glitch in spectrum or any indication of saturation from the verbal alarm log. Shortly afterward there was another smaller glitch in range, but this time with an ETMY saturation and glitch in spectrum.
21:48 UTC Peter and Jason done taking premodecleaner in car from OSB to LSB. They put it on a cart in the OSB receiving and wheeled it down the hallway to avoid opening the rollup door. I did not note the start time.
22:20 UTC Kyle and Gerardo to mid Y to valve out aux cart from ion pump.
22:30 UTC Kyle reports helicopter flying over end Y
23:01 UTC Kyle and Gerardo back. The ion pump was able to maintain the pressure on its own, so they turned off and disconnected the aux cart. All should be back to normal.