Evan, Kiwamu, Stefan

Evan reported that he observed a sudden drop in several AS poer signals, including 
H1:ASC-AS_C_SUM_OUT_DQ
H1:LSC-ASAIR_B_RF90_I_ERR_DQ
around Jul 31 2015 11:23:32 UTC

We found a corresponding drop of the 9MHz main oscillator feed, monitored by:
H1:ISC-RF_C_REFLAMP9M1_OUTPUTMON
H1:ISC-RF_C_AMP9M1_OUTPUTMON

attached is a file listing the channel differences between the L1 and H1 science frames.

During the period of bad IFO state which started at approx 12:12 PDT, the ITMX SWWD triped the SEI-B3 system. In the plot below, Ch1 shows the ITMX SUS top stage F1 OSEM signal, which rapidly exceeded its 95mV trip limit. This started the SUS SWWD counter. Five minutes later, Ch2 shows the signal going to the SEI-B3 drop to zero (the BAD state). This started the SEI SWWD counter, shown in Ch3 as going from one (GOOD) to three (1ST COUNTER). Four minutes later the SEI SWWD transitioned to four (2ND COUNTER) and one minute later tripped the SWWD, which zeroed all SEI-B3 DAC outputs. This shows up on the RMS plot as a slightly elevated signal, but SUS continues to be rung up. At 12:27 the operator intervened and manually panic'ed the SUS SWWD, at which point the RFM started decreasing.

Kyle, Gerardo

In and out of X-end VEA 

~1030 - 1230 hrs. local

Added 1.5" O-ring valve in-series with existing 1.5" metal angle valve -> Wrapped RGA with heater tapes and foil -> Elevated pump cart off of floor (resting on foam) -> NW40 inlet 50 L/s turbo (no vent valve) backed by aux. cart (no vent valve) -> Begin 100C bake 

In and out of X-end VEA between 

1405 - 1425 hrs. local, 

1450 - 1455 hrs. local and 

1600 - 1605 hrs. local. 

NOTE:  Will need to enter X-end VEA to make adjustments Saturday morning

LVEA: Laser Hazard
IFO: Locked
Observation Bit: Commissioning  

07:45 Cleared IPC errors on H1SUSTMSY & H1IOPASCO
08:00 IFO locked at Low Noise, 24.2w, 68Mpc 
08:35 Karen – Cleaning in the LSB cleaning area
09:16 LockLoss – Unknown
09:17 Kiwamu & Sudarshan – Going into LVEA to make ISS Outer Loop Servo measurements
09:19 Robert – Going into LVEA to setup for PEM injection
09:22 Jason – Going into LVEA to check PR3 OpLev
09:25 Kyle & Gerardo – Going to End-Y to get pump cart
09:30 ISI-ITMY WD trip – Robert doing PEM setup work in Beer Garden
09:53 Kyle & Gerardo – Finished at End-Y – back to CS
10:00 Kyle & Gerardo – Going to End-X to connect pump cart
10:17 Kyle – In LVEA to take valve off Pump Cart
10:20 Kyle – Out of LVEA
10:27 TJ – Going to End-X to reset BRS
11:07 Dave – Restarting models after OAF crash
11:12 Stefan – Going into MSR to work on atomic clock
11:27 Stefan – Out of MSR
11:30 Sudarshan – In LVEA plugging monitoring equipment
11:50 IFO Locked at Low Noise, 24.1w, 65Mpc
11:55 Dave – DAQ restart – NOTE: DAQ restart did not take IFO out of lock
12:12 IFO LockLoss – DAQ restart/Guardian/etc
12:36 Kyle & Gerardo – Back from End-X
12:39 Vendor on site to restock machines
12:47 Bubba – Test fire pumps
14:28 Dave & Stefan – At End-Y to install attenuator 
14:50 Dave & Stefan – Going to Mid-Y to power cycle PEM
15:05 Dave & Stefan – Back from Mid-Y
15:15 Dave & Stefan – Going to End-X to install attenuator
15:30 Dave & Stefan – Back from End-X

Today I performed the monthly fire pump test. All numbers and flows were acceptable.

Robert, Vinny, Stefan C, Dave:

The recently installed IRIGB signals were clipping the ADC due to the 10X gain in the PEM AA chassis. We installed a 10X attenuator in the line at the IRIG-B chassis in both end stations, the IRIGB signal range is now 0 - 5000 counts.

We also power cycled h1pemmy to see if this would remove the 64Hz noise being seen in the SEIS channels there. The procedure was: stop all models, power down h1pemmy, power down IO Chassis, power up IO Chassis, power up h1pemmy. We got lucky on the auto-code start and the timing came back OK with no further IOP restarts required.

Unfortunately this power cycle does not seem to have fixed the noise.

Scott L. Ed P. Rodney H.

7/30/15
The crew relocated lights, fans and cords to begin cleaning the next section starting at HSW-2-043. 51 meters of tube cleaned ending l2 meters east of HSW-2-041.
I repaired several of the extension cords and a cord on the vacuum used for cleaning the support tubes. 

7/31/15
Ed took today off. 
The support vehicles and all related equipment were relocated after lunch and a total of 61 meters of tube cleaned today ending 13.7 meters east of HSW-2-038.

The past ~200 meters of tube have been especially dirty with feces and urine. We have been giving these areas extra attention.

1239.5 meters of Y-Arm cleaned to date.

Stefan, Jim, Andres, Filiberto, Dave

WP5392

New h1calex and h1caley models were installed today, these read out ADC3-CHAN30 as the IRIGB signal. These signals were added to the commissioning frame and the frame broadcaster. The DAQ was restarted at 11:55 PDT to install this change.

Something happened at lock loss and the IFO did not reach the defined DOWN state.

Symptoms:

• SUS: ITMX, ITMY, ETMY - driven and rung up, watchdogs tripped, I had to adjust the RMS limit on ETMY from 8000 to 10000 to reset watchdog
• SEI: ITMY ISI tripped - Jim rest
• IO: MC WFS left pushing optics to a bad place and needed to be cleared by hand
• PSL: ISS and FSS in oscillation - outer loop on while IMC was unlock, throwing inner loop into oscillation
• GRD: ISC_LOCK - red for a while, cleared after INIT

Dave, Jaime, Sheila, operators, and others are investigating.

I ran BruCo on the quiet data period reported here. The report can be found at the following link:

https://ldas-jobs.ligo.caltech.edu/~gabriele.vajente/bruco_1122369690/

I'll look into the table and provide a summary later today.

Between 11:10 and 11:25 I adjusted the frequency offset and the phase offset in the timing diagnostic system's cesium clock in MSR so that the 1PPS time difference between it and our GPS backed timing distribution system equals 0±100ns and the long-term drift rate is near zero.

Last time I adjusted the timing offset (3 weeks ago), I subtracted 68e-15 from the factory-set frequency offset of 1078e-15; this was apparently the wrong direction, as evidenced by a very visible doubling in the cesium clock's drift rate in the weeks since that adjustment. Since the last measurement of drift rate was calculated from a month's worth of data and was ostensibly accurate, I went ahead and changed the frequency offset to (1078+68)e-15. The manufacturers instructions are vague on the correct adjustment direction and the tech support reps aren't very knowledgable, so I'm going to write up a short technical paper on Cs-III calibration with specific instructions for how to go about this.

I zeroed the phase using a 1PPS signal from the MSR master. The GPS timing jitter is O(10ns), which is well within the 100ns resolution of the CsIII phase syncing utility. I used a short (3') BNC cable to minimize delay.

I also went ahead and installed the most recent Monitor3 software on the Lenovo X61 Thinkpad in MSR and put the Cs-III instruction manual on that computer's desktop, so it should be easy for someone else to make finer adjustments to the frequency offset once we have another month or two worth of 1PPS drift data. 

at 10:37 PDT the IOP model on h1oaf0 received a timing error, all models stopped running at this time(h1pemcs, h1oaf, h1tcscs, h1odcmaster and h1calcs). The calculated IFO range went to a high value and the DARM FOM display went noisy. Only one external model receives IPC from the h1oaf0 computer, h1omc has a dolphin channel from the calcs, these channels went into 100% error.

There was PEM work in the CER around this time, this could have been the cause of the glitch. All the models were restarted and all is running correctly now. The intent bit was not set during this time.

BRS software crashed on the 25th, so today I went to EX to restrart the code. I kept the damper commented out, so the damper is currently OFF. I will turn it back on when it seems like it will be calm down at EX for a bit (vac team is down there right now).

Within a min afer a lockloss, the ISI ETMY ST1/2 WDs tripped. The medm said that it was ST1 T240 and the plots showed a slow drift to the WD trip level (plot attached). HEPI moved 170 um after lockloss before tidal began to bleed down. A further investigation is ongoing by the SEI team.

CDS: EE shop work to prep for Ring Heater installation
PSL: Wants to tweak PR3 OpLev at first available opportunity
FAC: Continued beam tube cleaning Y-Arm   
VAC: Moving pump cart to End-X for weekend long RGA bake out

Matt, Evan

Why do the TMSX RT and SD OSEMs have such huge spikes at 1821 Hz and harmonics? These spikes are about 4000 ct pp in the time series. In comparison, the other OSEMs on TMSX are 100 ct pp or less (F1 and LF shown for comparison).

Also attached are the spectra and time series of the corresponding IOP channels.