Power Outage LLO just experienced a power outage around 1350 UTC. https://alog.ligo-la.caltech.edu/aLOG/index.php?callRep=18564

Comments related to this report richard.oram@LIGO.ORG - 10:21, Monday 08 June 2015 (18568) Link Power outage @LIGO was of 5 seconds duration.

DEMCO control room informed us that the main feed from Watson to the LIGO substation was interrupted unexpectedly (possibly by a tree- they are sending crews out to investigate cause).

The automatic breakers detected the momentary short and re-established connection after five second pause. DEMCO does not expect any further interruption but will not guaranty it. We have started power outage recovery procedures.

When one makes a "comment" to an entry, there is a "Title" field (default-filled with "Comment to [original entry's title]"), but one could change the Title.  However, whatever is in the title field for the comment does not show up with the Comment entry.  It would be nice to tag your comment/sub-entry with Title.

Attached is the output from dmesg on h1broadcast0. It is reporting an divide error on the daqd process and suggesting a reboot. We should consider scheduling this for tomorrow's maintenance.

model restarts logged for Sun 07/Jun/2015
2015_06_07 03:51 h1fw1*
2015_06_07 06:28 h1fw1*
2015_06_07 19:18 h1fw1*

*=unexpected restart

0:02 Came to find IFO unlocked. AS Air shows 01 mode.

0:37 Lock loss at LOCK_DRMI_1F after trying to adjust PR3 and BS (the requested state not met)

0:41 Not trusting the alignment due to the 01 mode on AS Air I saw earlier, I restarted the initial alignment.

        ALS-C_COMM_A_DEMOD_RFMON was less than 5. I adjusted PR3 until the value reached 5.

        INPUT_ALIGN gives repeated message "arm not IR locked" and " timer[ 'pause' ] = 3". LSC-TR_X not flashing. No spot on AS Air.

1:35 Got off the phone with Sheila. She suggested SR2 and SR3 alignment might be bad. I realigned them to where they were 10 hours ago (when the IFO was still locked). It didn't solve the problem. Sheila then suggested I realign PR2 and IM4 as well.

         At this point the beam is back on AS Air, but LSC-TR_X still not flashing. So I realigned PR3, BS, ETMX, and ITMX back to where they were 10 hours ago. Lost ALS-X in the process.

3:58 Relocked ALS-X. INPUT_ALIGN request state finally met (*phew*). I tweaked PR2 YAW but couldn't get LSC-TR_X to reach 1 (it was ~0.97 -- close enough?).

         Adjusted BS during MICH_DARK_LOCK. The requested state was never met. I tweaked BS until ASAIR_B_RF90 went to 0 and moved on to the next state.

5:00 ALS_DIFF told me to find IR by hand. So I tweaked VCO until IR found. If you (fellow operators) ever have to do this, the narrow peak you see in LSC-TR_Y is a lie (in a sense that you can't get LSC-TR_Y to rise and become stable from there). Tweak VCO until you get a broad peak. Then decrease the step size and adjust the VCO until LSC-TR_Y is steady around 1.

5:20 Lock loss at BOUNCE_VIOLIN_MODE_DAMPING. No obvious reason. Rt. 10 traffic is picking up though. Lockloss analysis attached.

5:50 IFO at DC_READOUT -- waited until the IFO settle before requesting LSC_FF

5:55 Locking at LSC_FF (nominal). Range ~50 Mpc. Intent bit switched to undisturbed.

6:06 Sheila suggested earlier that I turn the ring heater power down to 0.4 W after the ifo is locked (from 0.5 W). So I change the RH power to 0.4 W. The intent bit is still undisturbed at this point.

8:00 Still locking and going strong. Handling off the ifo to Patrick (Covering for Cheryl).

If lock loss again during the day, I suggest whoever has the interferometer at that point redo the initial alignment. Maybe realign the optics I haven't touched or mentioned in this alog.  AS Air spot is off to the right and the range isn't that great (I've seen the ifo stable at a better range, though Vern is happy with where it is). The drift monitor shows that OMC pitch and yaw is "Somewhat out of range" and in "Danger Zone".

Intent bit switched to Undisturbed.

Times are local

20:00 John W called and said LVEA temps were running away, we agreed that LLO being down meant it was probably ok to risk turning off a heater

21:00 PI rings up, I turn power down at ~21:15, which messes with calibration

21:30 Lock loss, Guardian starts trying to lock, fixes change I made to laser power and thus fixes calibration

23:00 I start working on re-aligning after multiple RF DARM locklosses

Starting right around 21:00 local, the 25.4 hz (well, DTT says something like 25.37) peak showed back up, creating a nasty looking comb in the DARM spectra (first image). Unsure of what else to do, I turned the power down to 16 watts, the peak has now kind of subsided(second image). I'll wait to see if the peak settles down any more, then maybe turn the power back up. 

Reposted from LLO:

https://alog.ligo-la.caltech.edu/aLOG/index.php?callRep=18559

Peter Shawhan, Eric Thrane

We have restarted the burst injections at a rate of 1 / 2 hr. The latest test will check the stability of injection code tinj over >12hr following a bug fix. It will also facilitate gracedb testing. The current tests will use the same previously used burst sine-Gaussian waveform with scale factor = 4.0. Restart times:

LHO GPS = 1117767245
LLO GPS = 1117767486

It appears that the LVEA temperature is out of control. After consulting with Jim we decided to turn off a heater in the LVEA air handlers. This should bring the face bypass damper back in range.

HC2B was turned off. One heater, HC3A, remains on and will be turned off on Tuesday if tonight's change goes smoothly.

Plots show the LVEA temperature and the dampers which control airflow over the cooling coils. The dampers have railed the last two days in this hot weather.

This report is a copy-and-paste from here: https://alog.ligo-la.caltech.edu/aLOG/index.php?callRep=18555

Peter Shawhan, Chris Pankow, Dan Hoak, Eric Thrane

Dan noticed that the transient injection script died at LHO. Peter debugged and found two errors:

https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=18886
https://alog.ligo-la.caltech.edu/aLOG/index.php?callRep=18517

First, the code tinj was running out of file descriptors because of a missing fclose command, now fixed. There was also an error in the way the EPICS channel H1:CAL_TINJ_TYPE was set. It is now fixed to. Peter fixed the code in svn, but he could not recompile because of permission issues. (We are following up on that with the LHO CDS admins.) I recompiled the code at both sites and restarted the transient injection code:

GPS = 1117754506 (LHO)
GPS = 1117754609 (LLO)

There are currently no transient injections scheduled in the short-term future.

- Locked when I arrived.  Good range with some glitches until about 19UTC when braodband noise came up and reduced the range by 10-15Mpc.  

- Range did recover and then started to drop again, and then looked like it was going to recover and dropped lock.

- A blue pickup was seen on the camera about 5 minutes after lock loss - our landscapers, but not sure where they had been, only saw them head to the LSB.

- Relocking - had to start the ETMX ESD drive to lock, and then when IFO was locked again, Range was only 35Mpc.  I turned it off and Range retunrned to the level of the last lock.

- I reset all BS ISI watchdogs, and cleared the tripped watchdog on Stage 3.

- Evan returned CARM gain to a previous and higher value, and Range is just below 70Mpc.

- LLO just came back on the Range plot, and was down the last 8 hours.

- Currently: IFO locked with the Intent Bit engaged.

Description

See the attched figure.

The violin modes are not harmonics of each other, as we know. But they aren't harmonic in a rather interesting way, because the frequencies are less than harmonic. We might expect that the mode frequencies be greater than the harmonic values because of the bending stiffness of the fiber. Only a completely floppy string is harmonic. The fiber's modulus of elasticity will tend to become more important at higher mode numbers, increasing the frequencies.

However, we observe that the modes are lower in frequency than the harmonic values. The attached plot shows how close (or far) the modes are to harmonic. The y axis is the ratio of a mode's frequency to the first mode. The x axis is the mode number. An ideal harmonic system has a slope of 1. This is represented by the dashed black line. The blue line is from an FEA done by Alan Cumming at Glasgow University. This FEA takes into account the non-uniform shape of the fiber. The green line is from measurements of LHO ETMY.

Analysis

The FEA does indeed predict that the fiber should be less than harmonic. This could be because the 400 micron diameter fiber has 20 mm long 800 micron sections at the ends. Perhaps the effective bending point moves along the 800 micron section.

However, the measurements are even less harmonic than the FEA. It is not clear why this is. One possibility, though this sounds a bit crazy (but I can't think of anything else), is that the fiber is pitching the PUM and test mass as it oscillates. If the masses pitch, it makes the fiber behave as if it were longer, moving the frequencies down. Higher modes might exert more pitch torque becasue the fiber has more slope at its ends.

Running some numbers on the fiber length, The nominal value of 587 mm gives a fundamental frequency of 507 Hz, very close to ETMY (508 Hz) and within the spread of the other suspensions. But for ETMY mode 4, I have to increase the length by 20 mm. For ETMY mode 7 this is nearly 40 mm, which puts the bending point somewhere in the ears I think. For reference, the centers of mass are 602 mm apart.

Data

The mode frequencies in Hz used in the attached plot are 

FEA from Alan Cumming:

511, 1017, 1519, 2013, 2515, 3020, 3470

LHO ETMY:

508.3, 1009.1, 1484.5, 1956.7, 2425.8, 2880.6, 3333.2

The LHO ETMY data came from the following alogs

17610 - mode 1, averaged over the 4 given frequencies

17365 - modes 2 and 3. Mode 2 is averaged over the 4 given values. Mode 3 only has 1 value.

18764 - mode 4. This list doesn't specify which modes are which suspension. I chose to average the 4 highest values assuming they are ETMY because for all other modes ETMY is the highest.

18614 - modes 5, 6, and 7. I averaged the 2 given values for each of these modes.

I followed up the range drop tonight at 2015/06/07 around 8:30 UTC.

Here are the symptoms:
 - 25.4Hz and harmonics ringing up and saturating everything.
 - Significant increase of two lines: 842.781Hz and 868.188Hz (Yes, they are 25.407Hz apart) Those are marked with crosses in the attached plots.

Note that there are some other lines that increased (red) over the reference (black), but they are symmetric 25.4Hz sidebands of strong lines that did not increase. The two lines at 42.781Hz and 868.188Hz are not such modulation sidebands.

Parametric Instability?
alog 17903 reports on an observed parametric instability at 15540.6Hz, causing a line at 843.4Hz. Seems close enough to suspect it as the culprit for 842.781Hz. But what is 868.188Hz? And why does the 25.4Hz show up so strong?

BS roll mode?
My first though on seeing something at 25Hz was BS roll. But T1200415 reports the BS roll mode at 25.9715Hz... If we believe that, 25.4Hz can't be the BS roll...
Do we have an actual recent measurement of the BS roll?
I did try to look at the BS oplevs for a sign of the roll mode rung up - nothing. I haven't looked at the OSEMS yet

All attached plots were taken at 8:15 UTC, just before it go really bad. The black reference is from 7:00 UTC.

I also left instructions with Cheryl on how to lower power if this happens again. If that fixes it it would nail the PI.

Running of to the airport, but Cheryl will follow up.