Other activities in the area resulted in the CRYO-X ACC (H1:PEM-CS_ACC_LVEAFLOOR_XCRYO_Z_DQ) being knocked loose from the floor.  I cleaned the floor and ACC base and secured it with the PEM approved/special double-sided tape (seemed very secure.)  Also, the CRYO-Y instrument was checked and I found it attached but seeming very loose.  It was attached with a thick (1/32-1/16") foam double side tape which seemed old/deteriorated.  This attachment material is not approved!  I reattached it with the proper tape.

Attached are times series and spectra spanning the job.  The time-series are 3 days, the Tuesday am activities is obvious with the improved noise evident.  The spectra clearly reveal where the noise originates (fairly localized spots.  The REFs are the before.  Be mindful that I've been inconsiderate in that the DV and DTT plots are reversed in the X/Y top/bottom sense.  The unconnected and sort-of-attached spectra are quite different.  We may want to periodically inspect these instruments for security and make sure no unapproved tape has been used.

TITLE: 07/12 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Earthquake
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
    Wind: 21mph Gusts, 12mph 5min avg
    Primary useism: 0.04 μm/s
    Secondary useism: 0.06 μm/s 
QUICK SUMMARY: Problems with violin modes persist. Jeff K., Jenne, Hugh, et. al. attempting to address.

TITLE: 07/12 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Earthquake
INCOMING OPERATOR: Patrick
SHIFT SUMMARY:

H1 recovery continues.  First part of shift had HEPI Pump trip, then had issues getting to DC Readout due to a finkicky OMC.  Jenne & Kiwamu addressed the OMC & then Jeff Kissel, Hugh, & myself addressed many violin modes.  Jim took care of burdensome PI modes which came up.   So it has been a team effort.  Made it to Nominal_low_noise, but after skipping a few guardian steps.  While here for a couple hours, we worked on damping violins.

Most of the violin modes seemed to only need gain changes, but Jeff K. ran into a few modes which actually needed phase/sign changes.  So when the Guardian VIOLIN_MODE_DAMPING step kicks in, it has tweaked gains to old values and has caused us grief by ringing up violin modes.  

Jeff had some useful dtt templates for violin mode damping to use along with a striptool.  They are located here:

ligo/home/jeffery.kissel/Templates: 

• H1DARMIN1_forViolins.xml (fast one) 
• H1DCPDs_forViolins.xml (slow, higher resolution)

Recovery continues.

LOG:

• 15:34 Karen heading to end stations to install mouse traps
• 15:55 - 16:30: EX HEPI Pump Station Trip:  Hugh heading to investigate, most likely a voltage trip.
• 17:03  ENGAGE_SRC_ASC & Kiwamu removing OMC Whitening filters
  • saw POP90 & RF90 drift on StripTool, Kiwamu guessed it was SRM, so he zero-ed out dither gains for them and tried to tweak the SRM in yaw, but accidentally went the wrong way and broke lock.
• Fundamental violin mode (~500Hz) at just under e-12.  Had multiple people each working on different multiple modes for the next few hours.  
• 19:33 Liz brought in Moses Lake tour
• 20:20 Kiwamu going in LVEA checking impedence setting OMC DCPD
• 22:14 Lockloss due to ITMy MODE3 ringing up
• 22:38 Kyle to Xmid

Laser Status:
SysStat is good
Front End Power is 33.96W (should be around 30 W)
HPO Output Power is 154.6W
Front End Watch is GREEN
HPO Watch is GREEN

PMC:
It has been locked 1 days, 2 hr 56 minutes (should be days/weeks)
Reflected power = 16.33Watts
Transmitted power = 57.41Watts
PowerSum = 73.74Watts.

FSS:
It has been locked for 0 days 0 hr and 31 min (should be days/weeks)
TPD[V] = 2.702V (min 0.9V)

ISS:
The diffracted power is around 3.2% (should be 3-5%)
Last saturation event was 0 days 0 hours and 31 minutes ago (should be days/weeks)

Possible Issues:
Head 1-4 flow error, see SYSSTAT.adl

There was a brief, outdated section on seismic configuration on the SEI section of the OPS wiki, I've updated this with brief descriptions of each state in the SEI_CONF guardian. The wiki should be consistent with the table on ISI_CONFIG screen. I also removed a bunch of configurations that have been added temporarily during O2. I've also added a button to this page to the ISI_CONFIG screen, it's the big green button top-center of the screen. The button launches a firefox session that goes to : https://cdswiki.ligo-wa.caltech.edu/wiki/SEI . I'll try to keep the guide updated. Hopefully this will reduce confusion for everyone.

Last entry from Roam position 7, see 37334.

At Roam8, a meter or so further +X from Roam7, the spectra would suggest this location is not as quiet (wind response tilting) as Roam7.  However, this wind is from the NNW rather than our usual S or SW for previous views.  Also, while the Y dof seems more tilty for STS2-C (HAM5), at the lowest frequencies, HAM5 seems less tilty below 100mHz for the X dof.  I'm not sure if this conclusion is valid when the sensor is noisier during the quiet period???

Attached are the usual wind plot and sensor spectra.  The quiet wind time is 11 July 1200utc and the windy time begins 0230utc on 11 July.

So we have all these neon filled pods in the vacuum system and there was lots of discussion and concern about them but I don't think anything was ever done about that concern after installation.

It has been on my list to do something for some time but the differences in the BSC and HAM models slowed me down in implementing anything.  The BSCs have an epics setable widget 'level' which could be alarmed on but for some reason there is a reluctance to put an alarm on them.  Of course my 'alarm' is the old style EPICS alarm handler rather than the modern 'chatty cathy' or diag guardian.  Here nor there, at least at LHO, other than a green widget becoming red on the BSC Overview (see 1st attachment,) a pod could vent and we'd be none the wiser.  On the HAMs, maybe because the the HAMs 2 3 6 vs 4 5 differences, no alarm level infrastructure is implemented in the model.  The hams could be given an alarm system like the BSCs have but it would have to move from the master level to the individual level.  Of course, the alarming could just be put on the signals rather than relying on the model to calculate the color of the medm widget.

To the end of at least awareness of the Pods, I've modified an existing plotting script of JimWs to do this easily for inclusion in a FAMIS task.  Attached are the pod pressures and adjacent differentials for the ten ISI platforms.  The 'adjacent' is the vertical of a corner's horizontal/vertical pair.  The 'differential' is the vertical pod's analog pressure differenced with the horizontal pod pressure.  This scheme reduced the wires needed coming out of the pod.

These plots have just a single trace on each plot's graph.  I will look at other ways to present the traces to get problem sensors to stand out; such as, plot all the LVEA pod pressures on a single graph, and plot the ends and differentials on their own separate graphs keeping similar signals in a similar plenum and temperature environment together.  For now, just enjoy remembering these forgotten channels.

I have not set up the FAMIS job yet but the automated generating of the ten plots is a one button job: ISI_POD_PRESSURES on the WEEKLIES medm selection of the MAIN/OPS pulldown on the SITEMAP.

WP7066, ECR1700227 ASC 72/118 MHz WFS.

Kiwamu, Patrick, Dave:

a new h1asc model was installed. This added more fast and slow channels to the DAQ. It was quickly followed by a DAQ restart. Due to new RF72 names in the fast controls, but original RF90 names in slow controls, guardian's DIAG_MAIN required a change. Patrick rescanned the conlog channel list.

h1calex SDF reverted to monitoring CW GAIN and TRAMP

Paul, Dave:

h1calex_safe.snap (used for OBSERVE) was reverted to continue to monitor CW channels.

GDS code change, reboot of h1dmt0

Greg, Dave:

Greg updated the code on the GDS development machine (h1dmt3). For reason's unknown, later h1dmt0 developed problems and required a reboot to restore sensemon services.

h1guardian0 memory freed up by DIAG_MAIN restart

TJ, Dave:

At around 6am PDT TJ restarted guardian's DIAG_MAIN node, increasing the available memory on h1guardian0 from 17GB to 43GB. Later in the day I restarted DIAG_MAIN again due to ASC changes.

15:55 (9:55am) - 16:25:  While at RF_DARM, SEI & SUS Watchdogs tripped.   This was due to an EX HEPI Pump Controller Trip (Hugh notes it was:  "Over-voltage while constant running").  He went to EX to address & was mainly done down there by 16:10.

Jim jumped on damping everything that tripped at EX.  The ISI proved to be touchy & was bumped around quite a bit after being damped (after etmx & TMSX were re-damped);  so something mysteriously bumped the ISI.  So Jim brought the ISI back slowly, and we were were back after about 20min.

TITLE: 07/12 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Earthquake (since
OUTGOING OPERATOR: Owl was cancelled.
CURRENT ENVIRONMENT:
    Wind: 6mph Gusts, 4mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.05 μm/s
QUICK SUMMARY:

H1 was in the DOWN state overnight.  Slowly walking up through Guardian states.  ALS arms were mostly aligned only needing ETMy's pitch to be clicked a couple of times.  PRMI/DRMI locking was robust.  Will continue walking through alignment.

The following flow rates were measured with the calibrated flow meter.  All are in litres/min.
The data presented is: chiller display - Beckhoff chiller screen - measured flow rate - flow per crystal


22.6    22.6        17.46-17.71    4.40 (the current operating point)
21.3    21.4        16.60-16.85    4.18
24.5    24.6-24.7   19.30-19.53    4.85
26.0    26.0        20.56-20.78    5.17
28.0    28.0-28.1   22.21-22.48    5.59
29.8    29.7        23.56-23.95    5.94





  JeffB / Peter

TITLE: 07/11 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Earthquake (since I believe this is still recovery from the earthquake in Montana)
INCOMING OPERATOR: Canceled (would have been TJ)
SHIFT SUMMARY: Jeff K. and I have talked with Vern and we agreed to call it a night. The remainder of this shift and TJ's owl shift have been canceled. I have notified TJ, Corey and LLO. I have set ISC_LOCK to DOWN for the night. I believe Jeff K. speculated that something may be wrong with the AS signal. The last lock loss occurred after the appearance of a violin mode or modes at around 501 Hz that were not there in the previous lock. I believe everyone except Thomas V. has left. I may stay to work on other projects.
LOG:

23:19 UTC The OMC_LOCK guardian was not completing. Jenne pointed out that the OMC ASC was saturating (Sitemap -> OMC -> OMC Control -> bottom right outputs). She had me click 'Graceful clear history' (in purple on the same screen). Once the outputs went to 0 I re-requested READY_FOR_HANDOFF and the OMC_LOCK guardian completed successfully.
23:52 UTC Jeff K. turning off excess damping. Lock loss on transition from DC_READOUT_TRANSITION to DC_READOUT.
23:56 UTC Kiwamu to CER to plug in RF90 sensor
00:08 UTC I changed the Ops Observatory mode from Aligning to Lock Acquisition and then to Earthquake since this may still be recovery from the earthquake in Montana.
00:14 UTC Stopped at DC_READOUT_TRANSITION. Stepping through by hand. Lost lock on loading matrix to transition to OMC.
00:30 UTC Stopped at DC_READOUT_TRANSITION. Jenne and Kiwamu investigating.
00:34 UTC Robert and Pep to end Y VEA. Looking for lines from RF card reader and WIFI.
01:00 UTC Robert and Pep back.
01:01 UTC Kiwamu didn't find anything to account for the lockloss on transition to DC_READOUT. Tried again. Worked this time.
01:12 UTC Lock loss while damping 509 Hz violin mode while sitting at LOWNOISE_ASC.
01:38 UTC Lock loss sitting at DC_READOUT_TRANSITION while attempting to lock OMC.
01:52 UTC Lock loss at ENGAGE_SOFT_LOOPS. Appearance of violin mode at around 501.75 Hz. Not rung up in previous locks.
02:03 UTC Set ISC_LOCK to DOWN and leaving it for the night.

J. Kissel, E. Merilh, P. Thomas, T. Vo, J. Driggers, K. Izumi,

Last night ETMX violin modes 507.159 and 507.194 Hz had mysteriously rung up on Patrick last night, and unfortunately in his attempts to do something about he rung up the mode to all get out (visible way above even an RF DARM spectra). 

After maintenance was complete (around 1:00p local), we were able to smoothly get up ENGAGE_SRC_ASC -- but only after
 -  an initial alignment 
 - a discovery of the YAW input to PRM being off from the ASC model reboot today (Thanks DIAG MAIN!!), and
 - a wrong POP QPD A offset, also from the ASC model reboot (thanks to Kiwmau's detective work).
 
Once that was figured out, I spent the afternoon finding settings of the ETMX MODE6 and MODE7 violin mode damping filter banks that would *damp* the modes instead of exciting them. In the end, what worked was a change *only* in the MODE7 bank: 
    use negative gain, and only FM1 (507.194New) and FM4(100dB) .
(as opposed to what's encoded in guardian and in the violin mode table as positive gain and FMs 1(507.194New), 3(+60deg), and 4(100dB))
With these settings, it was the usual game of watching the control outputs and slowly increasing the gains to preserve one's sanity. 
I also had initial success by *only* using MODE7, and keeping MODE6 low -- this was to get the 507.194 mode down enough below the 507.159 mode that things started behaving more normally (with more gain on MODE6 than 7).

Once all ETMX modes were damped to the noise floor of AS_Q ("RF DARM") -- including extra gain on ETMX MODES 5 and 8 at 506.922 and 507.391 once the above middle two were under control -- and confirmed that neither DCPD were close to saturation, we tried transitioning to DC readout.

This failed on us several times, exactly when the DARM error signal was switched over to the DCPDs.
We explored several reasons as to why this might be going wrong (hanging out in DC_READOUT_TRANSITION), but in the end we only found that the AS_AIR and OMC READOUT ERR signals (which should be ideally unity and flat in this state) had the attached, very weird transfer function (though in truth this is a new lock, the DC gain was only -10 dB in the prior lock).
Even so, we were able to transition that third time.

We're still exploring the fishiness, but quickly running out of steam...

The production DMT computer, h1dmt0, was hung. (I could not do an "ls" on the DMT monitor logs directory, and SenseMonitor_H1
and H1_llhoft frames had stopped showing up. This was perhaps related to the mornings DAQ reboot, as the SenseMonitor_H1 frames stopped showing up shortly after this. The H1_llhoft frames stopped around the time of the morning update of gds on h1dmt3, though supposedly this should not have caused any problems on h1dmt0.)

After a discussion with Dave Barker, a reboot of h1dmt0 seems to have fixed things.

Greg is looking into problems with h1dmt0, which is causing non-reporting of  H1's range.

WP7066 Kiwamu, Dave:

a new h1asc model was installed, replacing RF90 with RF72. The DAQ was restarted soon afterwards.

DAQ Changes:

Fast Channels Added (2048Hz)

+: fast channel H1:ASC-AS_A_RF72_I_PIT_OUT_DQ added to the DAQ
+: fast channel H1:ASC-AS_A_RF72_I_SUM_OUT_DQ added to the DAQ
+: fast channel H1:ASC-AS_A_RF72_I_YAW_OUT_DQ added to the DAQ
+: fast channel H1:ASC-AS_A_RF72_Q_PIT_OUT_DQ added to the DAQ
+: fast channel H1:ASC-AS_A_RF72_Q_SUM_OUT_DQ added to the DAQ
+: fast channel H1:ASC-AS_A_RF72_Q_YAW_OUT_DQ added to the DAQ
+: fast channel H1:ASC-AS_B_RF72_I_PIT_OUT_DQ added to the DAQ
+: fast channel H1:ASC-AS_B_RF72_I_SUM_OUT_DQ added to the DAQ
+: fast channel H1:ASC-AS_B_RF72_I_YAW_OUT_DQ added to the DAQ
+: fast channel H1:ASC-AS_B_RF72_Q_PIT_OUT_DQ added to the DAQ
+: fast channel H1:ASC-AS_B_RF72_Q_SUM_OUT_DQ added to the DAQ
+: fast channel H1:ASC-AS_B_RF72_Q_YAW_OUT_DQ added to the DAQ

Fast Channels Removed (2048Hz)

-: fast channel H1:ASC-AS_A_RF90_PIT_OUT_DQ removed from DAQ
-: fast channel H1:ASC-AS_A_RF90_YAW_OUT_DQ removed from DAQ
-: fast channel H1:ASC-AS_B_RF90_PIT_OUT_DQ removed from DAQ
-: fast channel H1:ASC-AS_B_RF90_YAW_OUT_DQ removed from DAQ

Slow Channel Stats: 1060 channels Added, 356 channels Removed

J. Kissel, H. Radkins, J. Warner

We're continuing to attack violin modes post Montana EQ. I attach a spectrum of "our" (all Jim and Hugh) progress, where the reference (GREEN) is this morning. We can definitely see that the violin modes begin to get hefty non-linear shoulders by the time the RMS passes ~5e2 ADC counts, which is pretty surprising for a 32e4 count ADC...

New modes that Hugh and Jim have now successfully cooled today:

EX      1475.09       MODE7     YAW      FM4(100dB), FM6(1475.09),                  G = -500
      (This mode interacts with the "already solved" 1475.25 EX mode, so that damping must also be ON and some small level while damping this mode)
IY      1472.45       MODE8     PIT      FM4(100dB), FM6(1462.45), FM7(-60d1.5k)    G = +1000
IY      1470.38       MODE4     PIT      FM4(100dB), FM6(1470.38),                  G = -500
IY      1470.82       MODE5     PIT      FM4(100dB), FM6(1470.82), FM7(-60d1.5k)    G = +1000
As before -- these are reported for the record, but the Violin Mode Table has been updated and that should be treated as cannon not this aLOG. 

I've been continuing to manipulate the damping output of the problematic pair of modes at 1463.09 Hz, including
  - Installing narrow bandpass in all test masses, exploring the unit circle with each of the 4 test masses, and all 3 degrees of freedom (L, P, Y)
  - Went back to ITMX and focused energy there (since that;s what I was driving when I originally rung it up)
  - Adding +/- 30 [deg] filters, and spinning around the unit circle there
  - Broadening the band pass, in hopes of changing the relative drive phase between the two modes.
After three hours of failure, I was *just* about to think I saw hints of progress driving the wide-band band pass in YAW, with FM4(100dB) and FM5(1463.09W) and +1 gain, but then we lost lock because the PSL tripped. Guh!

While wasting time staring at StripTool for 3 hours, I grabbed a 0.5 [mHz] resolution spectra to confirm the violin mode table's claimed exact frequency of these modes. Unfortunately, DTT (and foton's) cursor can't resolve past 2 decimal places. But at least zooming in visually, I can confirm that the two modes are:
    1463.0961 +/- 0.0005
    1463.0982 +/- 0.0005

We were focused on the 1.5k modes so far today, and made no attempts at progress on the remaining high 1.0k pair of modes at 1008.45 / 1008.49 Hz.

Description

See the attched figure.

The violin modes are not harmonics of each other, as we know. But they aren't harmonic in a rather interesting way, because the frequencies are less than harmonic. We might expect that the mode frequencies be greater than the harmonic values because of the bending stiffness of the fiber. Only a completely floppy string is harmonic. The fiber's modulus of elasticity will tend to become more important at higher mode numbers, increasing the frequencies.

However, we observe that the modes are lower in frequency than the harmonic values. The attached plot shows how close (or far) the modes are to harmonic. The y axis is the ratio of a mode's frequency to the first mode. The x axis is the mode number. An ideal harmonic system has a slope of 1. This is represented by the dashed black line. The blue line is from an FEA done by Alan Cumming at Glasgow University. This FEA takes into account the non-uniform shape of the fiber. The green line is from measurements of LHO ETMY.

Analysis

The FEA does indeed predict that the fiber should be less than harmonic. This could be because the 400 micron diameter fiber has 20 mm long 800 micron sections at the ends. Perhaps the effective bending point moves along the 800 micron section.

However, the measurements are even less harmonic than the FEA. It is not clear why this is. One possibility, though this sounds a bit crazy (but I can't think of anything else), is that the fiber is pitching the PUM and test mass as it oscillates. If the masses pitch, it makes the fiber behave as if it were longer, moving the frequencies down. Higher modes might exert more pitch torque becasue the fiber has more slope at its ends.

Running some numbers on the fiber length, The nominal value of 587 mm gives a fundamental frequency of 507 Hz, very close to ETMY (508 Hz) and within the spread of the other suspensions. But for ETMY mode 4, I have to increase the length by 20 mm. For ETMY mode 7 this is nearly 40 mm, which puts the bending point somewhere in the ears I think. For reference, the centers of mass are 602 mm apart.

Data

The mode frequencies in Hz used in the attached plot are 

FEA from Alan Cumming:

511, 1017, 1519, 2013, 2515, 3020, 3470

LHO ETMY:

508.3, 1009.1, 1484.5, 1956.7, 2425.8, 2880.6, 3333.2

The LHO ETMY data came from the following alogs

17610 - mode 1, averaged over the 4 given frequencies

17365 - modes 2 and 3. Mode 2 is averaged over the 4 given values. Mode 3 only has 1 value.

18764 - mode 4. This list doesn't specify which modes are which suspension. I chose to average the 4 highest values assuming they are ETMY because for all other modes ETMY is the highest.

18614 - modes 5, 6, and 7. I averaged the 2 given values for each of these modes.