TITLE: 07/13 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Earthquake
INCOMING OPERATOR: Corey
SHIFT SUMMARY: After the earthquake I relocked back to LOWNOISE_ASC and skipping VIOLIN_MODE_DAMPING_1. I then tried to get through all of the modes and checking with the wiki and Guardian. I got a bit confused as to what changes were recent and what were old. The wiki had "changed July 12" but no year, and going back to some old versions of the wiki before the 12 they were still there so I assume they are old. Unless it wasn't actually showing me an older version. Either way, below is a list of the progress I made on the violin modes.

ITMX

• Modes 1-8 from the wiki are good.

ITMY

• Modes 3,6,7 from the wiki are good
• 8 works but is very slow
• 5 a +200 gain gave me faster results but still slower

ETMX

• 1,3,7 good
• 2, the only thing I was able to very slowly work was FM4&5 -100 gain
• 4 Couldn't get anything to consistently work. I would try one setting and get results only to have it turn around on me later and then not work when I went to try it later.

ETMY

• I mainly just worked on Modes 4 and 5. These are really close in frequency, 508.206hz and 508.220hz, so they seem to be a bit coupled. I spent a lot of time thinking I had a good setting for these, only to have it ring up on me tens of minutes after slowly damping. I could never damp both very well, it was only one or the other, and it was very sensitive to what gain I put in. putting in the wrong gain into any one of the two would ring up both and I'd have to start from scratch. My best result was damping mode 4 and very very slightly mode 5 with:
  • Mode 4 => FM1,3,4   -20 gain
  • Mode 5 => FM1,3,4   -50 gain

If I didn't mention a mode above, I either didn't get to it, or forgot to write it down.

Odd lock loss. Zero signs of anything going on and both stages of the BS ISI tripped. Plots of the CPS are kinda sudden, I haven't seen that before.

As I was trying to lock arms green the platform tripped again... The platform tripped while turning on isolation loops, I hadn't gotten to locking arms green yet.

Added 50mL to the Xtal chiller

Relocked with no problems, sitting at LOWNOISE_ASC. I turned on all the ITMX damping since patrick said that it was good, as well as ITMY mode 3,6,7 and ETMX mode 7. Much of the last few hours has been playing with ETMY modes 4 & 5. I will try something and it will look like it works for tens of minutes, then it will just start to slowly ring up, then a bit faster. I've tried a few different settings and all with varying, but similar, results with a time delay of ringing it up.

TITLE: 07/13 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Earthquake
OUTGOING OPERATOR: Patrick
CURRENT ENVIRONMENT:
    Wind: 10mph Gusts, 8mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.06 μm/s
QUICK SUMMARY: Earthquake has rung down and it's time for initial alignment. We'll see where the violins are at after.

TITLE: 07/12 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Earthquake
INCOMING OPERATOR: TJ
SHIFT SUMMARY: Things were going relatively well until an earthquake in Papua New Guinea knocked us out of lock. I had just finished setting all the ITMX violin mode damping settings for the first fundamentals to the values in the wiki table and had them damping successfully. I had been able to get the first fundamental of ETMY mode 4 to damp using the settings from the wiki table, but was not having much luck with modes 5 and 6. The ETMX, ITMX and ITMY watchdogs tripped when the earthquake hit. I set the ISI config to LARGE_EQ_NOBRSXY while they were tripping, it may have saved other tables. The BLRMS has almost come back down to nominal and I have put the ISI config back to WINDY. I was able to lock both arms on green, but the X arm is only around .89. I will suggest to TJ that he does an initial alignment. Rick called at 5:50 UTC to tell me he is working in his office.

LOG:

00:06 UTC Kyle back from mid station
05:30 UTC Reset ETMX ISI watchdogs
05:33 UTC Reset ITMX ISI watchdogs
05:36 UTC Reset ITMY ISI watchdogs
06:09 UTC Trying locking arms green. BLRMS is ~ .2 in 0.03 - 0.1 Hz band
Y arm locks, X arm not so much.
06:33 UTC Setting ISI config back to WINDY
06:35 UTC Cleared WFS history for X arm ALS. Got X arm to lock at ~.88. I think it is going to need an initial alignment.

6.4 and/or 6.7 in Papua New Guinea

Verbal alarm notification of incoming earthquake at 03:43 UTC.
Lockloss at 04:19:30 UTC.

ISI ETMX Stage 2
ISI ETMX Stage 1
ISI ITMX Stage 2
ISI ITMY Stage 2
ISI ITMX Stage 1
ISI ITMY Stage 1
all tripped before I managed to transition to LARGE_EQ_NOBRSXY.

I had just finished slowly setting all the ITMX violin mode settings for the first fundamentals to the values in the violin mode table, which all seemed to work.

Sitting at LOW_NOISE_ASC and damping violin modes. Gave up on ETMY mode 5 and 6 for now (set their gains to 0), moving over to ITMX.

J. Kissel, J. Driggers, P. Thomas, C. Gray, H. Radkins

After beating down yet another 500 Hz mode that had gone bananas we've now resorted turning off all automated damping (by skipping those steps in the lock-acquisition sequence). Further, (and what I think may have also contributed to the badness & confusion), we've turned off the automatic adjustment of gains during the VIOLIN_MODE_DAMPING state. 

For the record, I think all 1 kHz, 1.5 kHz and high harmonics are all damped enough or have rung down themselves enough that these are no longer a problem. Our remaining problems is fixing the automated damping.

So now we're going through, by hand, and checking that either the modes are damped with what's reported in the violin mode table, or recording the difference. So far we've found three modes that are different:
  IY MODE3 501.749 FM1 4     -5G
  IY MODE6 503.120 FM1 3 4   +200G
  EX MODE7 507.194 FM1 4     -50G

Jenne has programed these new settings into the guardian and loaded it. THESE HAVE NOT YET BEEN UPDATED IN THE VIOLIN MODE TABLE.

However -- we recommend that the remainder of the night, and into the owl shift, be spent carefully turning on all 500 Hz damping filters, and either confirming that the violin mode table is right, or finding new settings (phases / signs). Be cautious with the gain for starters, with the settings as in the violin mode table, but be watching both spectra and control outputs. And BE PATIENT. Wait 5 minutes or so to see whether your change has done good. A lot of the troublesome modes are close together, so the beatnotes between close modes can be deceiving.

Other activities in the area resulted in the CRYO-X ACC (H1:PEM-CS_ACC_LVEAFLOOR_XCRYO_Z_DQ) being knocked loose from the floor.  I cleaned the floor and ACC base and secured it with the PEM approved/special double-sided tape (seemed very secure.)  Also, the CRYO-Y instrument was checked and I found it attached but seeming very loose.  It was attached with a thick (1/32-1/16") foam double side tape which seemed old/deteriorated.  This attachment material is not approved!  I reattached it with the proper tape.

Attached are times series and spectra spanning the job.  The time-series are 3 days, the Tuesday am activities is obvious with the improved noise evident.  The spectra clearly reveal where the noise originates (fairly localized spots.  The REFs are the before.  Be mindful that I've been inconsiderate in that the DV and DTT plots are reversed in the X/Y top/bottom sense.  The unconnected and sort-of-attached spectra are quite different.  We may want to periodically inspect these instruments for security and make sure no unapproved tape has been used.

TITLE: 07/12 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Earthquake
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
    Wind: 21mph Gusts, 12mph 5min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.06 μm/s 
QUICK SUMMARY: Problems with violin modes persist. Jeff K., Jenne, Hugh, et. al. attempting to address.

TITLE: 07/12 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Earthquake
INCOMING OPERATOR: Patrick
SHIFT SUMMARY:

H1 recovery continues.  First part of shift had HEPI Pump trip, then had issues getting to DC Readout due to a finkicky OMC.  Jenne & Kiwamu addressed the OMC & then Jeff Kissel, Hugh, & myself addressed many violin modes.  Jim took care of burdensome PI modes which came up.   So it has been a team effort.  Made it to Nominal_low_noise, but after skipping a few guardian steps.  While here for a couple hours, we worked on damping violins.

Most of the violin modes seemed to only need gain changes, but Jeff K. ran into a few modes which actually needed phase/sign changes.  So when the Guardian VIOLIN_MODE_DAMPING step kicks in, it has tweaked gains to old values and has caused us grief by ringing up violin modes.  

Jeff had some useful dtt templates for violin mode damping to use along with a striptool.  They are located here:

ligo/home/jeffery.kissel/Templates: 

• H1DARMIN1_forViolins.xml (fast one) 
• H1DCPDs_forViolins.xml (slow, higher resolution)

Recovery continues.

LOG:

• 15:34 Karen heading to end stations to install mouse traps
• 15:55 - 16:30: EX HEPI Pump Station Trip:  Hugh heading to investigate, most likely a voltage trip.
• 17:03  ENGAGE_SRC_ASC & Kiwamu removing OMC Whitening filters
  • saw POP90 & RF90 drift on StripTool, Kiwamu guessed it was SRM, so he zero-ed out dither gains for them and tried to tweak the SRM in yaw, but accidentally went the wrong way and broke lock.
• Fundamental violin mode (~500Hz) at just under e-12.  Had multiple people each working on different multiple modes for the next few hours.  
• 19:33 Liz brought in Moses Lake tour
• 20:20 Kiwamu going in LVEA checking impedence setting OMC DCPD
• 22:14 Lockloss due to ITMy MODE3 ringing up
• 22:38 Kyle to Xmid

Laser Status:
SysStat is good
Front End Power is 33.96W (should be around 30 W)
HPO Output Power is 154.6W
Front End Watch is GREEN
HPO Watch is GREEN

PMC:
It has been locked 1 days, 2 hr 56 minutes (should be days/weeks)
Reflected power = 16.33Watts
Transmitted power = 57.41Watts
PowerSum = 73.74Watts.

FSS:
It has been locked for 0 days 0 hr and 31 min (should be days/weeks)
TPD[V] = 2.702V (min 0.9V)

ISS:
The diffracted power is around 3.2% (should be 3-5%)
Last saturation event was 0 days 0 hours and 31 minutes ago (should be days/weeks)

Possible Issues:
Head 1-4 flow error, see SYSSTAT.adl

There was a brief, outdated section on seismic configuration on the SEI section of the OPS wiki, I've updated this with brief descriptions of each state in the SEI_CONF guardian. The wiki should be consistent with the table on ISI_CONFIG screen. I also removed a bunch of configurations that have been added temporarily during O2. I've also added a button to this page to the ISI_CONFIG screen, it's the big green button top-center of the screen. The button launches a firefox session that goes to : https://cdswiki.ligo-wa.caltech.edu/wiki/SEI . I'll try to keep the guide updated. Hopefully this will reduce confusion for everyone.

Last entry from Roam position 7, see 37334.

At Roam8, a meter or so further +X from Roam7, the spectra would suggest this location is not as quiet (wind response tilting) as Roam7.  However, this wind is from the NNW rather than our usual S or SW for previous views.  Also, while the Y dof seems more tilty for STS2-C (HAM5), at the lowest frequencies, HAM5 seems less tilty below 100mHz for the X dof.  I'm not sure if this conclusion is valid when the sensor is noisier during the quiet period???

Attached are the usual wind plot and sensor spectra.  The quiet wind time is 11 July 1200utc and the windy time begins 0230utc on 11 July.

So we have all these neon filled pods in the vacuum system and there was lots of discussion and concern about them but I don't think anything was ever done about that concern after installation.

It has been on my list to do something for some time but the differences in the BSC and HAM models slowed me down in implementing anything.  The BSCs have an epics setable widget 'level' which could be alarmed on but for some reason there is a reluctance to put an alarm on them.  Of course my 'alarm' is the old style EPICS alarm handler rather than the modern 'chatty cathy' or diag guardian.  Here nor there, at least at LHO, other than a green widget becoming red on the BSC Overview (see 1st attachment,) a pod could vent and we'd be none the wiser.  On the HAMs, maybe because the the HAMs 2 3 6 vs 4 5 differences, no alarm level infrastructure is implemented in the model.  The hams could be given an alarm system like the BSCs have but it would have to move from the master level to the individual level.  Of course, the alarming could just be put on the signals rather than relying on the model to calculate the color of the medm widget.

To the end of at least awareness of the Pods, I've modified an existing plotting script of JimWs to do this easily for inclusion in a FAMIS task.  Attached are the pod pressures and adjacent differentials for the ten ISI platforms.  The 'adjacent' is the vertical of a corner's horizontal/vertical pair.  The 'differential' is the vertical pod's analog pressure differenced with the horizontal pod pressure.  This scheme reduced the wires needed coming out of the pod.

These plots have just a single trace on each plot's graph.  I will look at other ways to present the traces to get problem sensors to stand out; such as, plot all the LVEA pod pressures on a single graph, and plot the ends and differentials on their own separate graphs keeping similar signals in a similar plenum and temperature environment together.  For now, just enjoy remembering these forgotten channels.

I have not set up the FAMIS job yet but the automated generating of the ten plots is a one button job: ISI_POD_PRESSURES on the WEEKLIES medm selection of the MAIN/OPS pulldown on the SITEMAP.

WP7066, ECR1700227 ASC 72/118 MHz WFS.

Kiwamu, Patrick, Dave:

a new h1asc model was installed. This added more fast and slow channels to the DAQ. It was quickly followed by a DAQ restart. Due to new RF72 names in the fast controls, but original RF90 names in slow controls, guardian's DIAG_MAIN required a change. Patrick rescanned the conlog channel list.

h1calex SDF reverted to monitoring CW GAIN and TRAMP

Paul, Dave:

h1calex_safe.snap (used for OBSERVE) was reverted to continue to monitor CW channels.

GDS code change, reboot of h1dmt0

Greg, Dave:

Greg updated the code on the GDS development machine (h1dmt3). For reason's unknown, later h1dmt0 developed problems and required a reboot to restore sensemon services.

h1guardian0 memory freed up by DIAG_MAIN restart

TJ, Dave:

At around 6am PDT TJ restarted guardian's DIAG_MAIN node, increasing the available memory on h1guardian0 from 17GB to 43GB. Later in the day I restarted DIAG_MAIN again due to ASC changes.

15:55 (9:55am) - 16:25:  While at RF_DARM, SEI & SUS Watchdogs tripped.   This was due to an EX HEPI Pump Controller Trip (Hugh notes it was:  "Over-voltage while constant running").  He went to EX to address & was mainly done down there by 16:10.

Jim jumped on damping everything that tripped at EX.  The ISI proved to be touchy & was bumped around quite a bit after being damped (after etmx & TMSX were re-damped);  so something mysteriously bumped the ISI.  So Jim brought the ISI back slowly, and we were were back after about 20min.

TITLE: 07/12 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Earthquake (since
OUTGOING OPERATOR: Owl was cancelled.
CURRENT ENVIRONMENT:
    Wind: 6mph Gusts, 4mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.05 μm/s
QUICK SUMMARY:

H1 was in the DOWN state overnight.  Slowly walking up through Guardian states.  ALS arms were mostly aligned only needing ETMy's pitch to be clicked a couple of times.  PRMI/DRMI locking was robust.  Will continue walking through alignment.