J. Kissel, E. Merilh, J. Driggers, K. Izumi, D. Barker

Just on an update on the recovery process this morning:
- ~12:30 UTC Giant EQ in Alaska (LHO aLOG 35930)
    - EQ is on our tectonic plate, so happens too fast for Bartlett to switch ISI Systems to Robust / EQ mode, so they all trip
- ~14:30 UTC Giant Aftershock (LHO aLOG 35931)
    - Same story: EQ is on our tectonic plate, so happens too fast for Bartlett to switch ISI Systems to Robust / EQ mode, so they all trip
- ~15:00 UTC Ed Merilh takes over from Bartlett
- 16:30 UTC Ed having trouble with INITIAL ALIGNMENT (specifically with PRC WFS loops would not close)
    - Solution was to skip to MICH_DARK_LOCKED and make sure BS was well aligned, and then back to PRC and move PRM in pitch until AS port looked well
- ~17:30 UTC Finished with Initial Alignment, microseism and EQ band have sufficiently rung down to begin locking
- ~18:00 UTC Found that we couldn't get past LOCKING ALS because ETMY Bounce Mode, spent a LONG time running with ALS_DIFF at a factor of 10 less gain, and SLLOOOOWWWWLYY damping the mode with gains of 0.001, and we eventually got up to 0.028, and then
- ~20:05 UTC The SEI BS crashed.

Since we're already down, we voted for just completing all corner station and end station front-end reboots.
Dave's working on that now, more news on everything later.

h1seib2 froze up at 13:15 local time, we are now restarting all machines with more than 208 days of run time.

(manual run of the scripts, crontabs were not operational)

Starting CP3 fill. LLCV enabled. LLCV set to manual control. LLCV set to 50% open. Fill completed in 40 seconds. TC B did not register fill. LLCV set back to 18.0% open.

Starting CP4 fill. LLCV enabled. LLCV set to manual control. LLCV set to 70% open. Fill completed in 630 seconds. TC A did not register fill. LLCV set back to 42.0% open.

CP3 log file DOES NOT exist!
CP4 log file DOES NOT exist!

The regular Monday 11am LN2 auto-overfill of CP3 and CP4 did not run this morning, presumably related to OS updates on Friday. Will run it manually for today.

Having moved the STS2-C (HAM5) from Roam2 (near NE BSC8{H2 ITMY} chamber leg/SEI Pier) to Roam3 (~2/3 distance to BSC4 {H2 BS}) on Friday during EQ downtime, some weekend wind has given a good comparison to examine.  Reference Traces are from the quiet period (0805utc May 1) and the current traces are  during the windy time beginning at 2300utc April 30.

See the first plot attached showing the wind velocity at the corner station.  180 degrees on the anemometer is pretty much right out of the +X to -X direction, i.e., no Y component and this may likely explain why the Y dof comparison of the ASD is nearly identical.  The coherence shows stronger frequency dependence.  On the X dof, the STS2-C at Roam3 is certainly noisier than STS2-B at its home location and a bit more so during the higher wind.  However, comparing the same plots from aLog 35848, Roam3 is a much less noisier location than Roam2.  Caveat: the wind may be a bit quieter (few mph) and the direction is nearly opposite.  But, we might feel the largest deal is ~doubling the distance from the south wall.  Still, the North wall which is getting the direct impact of the wind here is much farther (at least 3 times farther) than the South wall which was getting the direct hit from the aLog 35848 wind storm.  On the Z dof comparisons, the low wind time equity extends to lower frequencies and during high wind that breaks down around 10 or 20mHz.

Conclusion, Roam3 is a better position than Roam2 location, at least for this wind direction but the STS2-B location remains best.  Need a good SW wind (from +Y) to really fill in some holes.

We had two more lock ups over the weekend. Luckily they were on non-dolphined corner station machines (SUS-AUX) and I was able to work remotely with the operator on getting these machines reset via the front panel reset button.

Here is a summary of the lockups since Wednesday night:

John Zweizig has fixed the problem reported here,

https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=35924

which was caused by the DMT RAID box running out of inodes.

John has fixed the problem by deleting older data to free inodes.

TITLE: 05/01 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Environment
OUTGOING OPERATOR: Jeff
CURRENT ENVIRONMENT:
    Wind: Calm
    Primary useism: EQ
    Secondary useism: EQ
QUICK SUMMARY: ops lazy script isn't doing transition (-t).

Shift Summary: Run A2L check script. Pitch is below reference, Yaw a bit elevated. Received GRB. Both sites in observing. Spoke with LLO. In one hour hold until 12:12 (05:12).

FE computer H1IOPSUSAUXB123 crashed. Spoke to Dave B. – Reset FE computer to recover.  Back into observing.

Received an EQ warning and shortly lost lock. USGS showing a Mag6.5 in Alaska. Both Primary and Secondary microseism are off the top of the charts. Tripped the WD on most SUS, all, ISIs, several HEIPs, and all four IMs. Have the IFO in DOWN until things settle down a bit. Have been resetting tripped WD as necessary. A dozen after socks in Alaska have been recorded. Primary and Secondary microseism are still above 1.0um/s.

As the seismic was ringing down tried relocking. After relocking ALS Green, hit with another Mag6.3 earthquake near Skagway Alaska. Seismic Primary and Secondary are off the top of the charts. Put the IFO back into DOWM and started resetting the SUS and SEI watchdogs.               

6.5 Magnitude EQ Near Skagway, Alaska at 12:31utc, (05:31PT).

• Arrival ~12:40utc (05:40PT)
• Seen on USGS, Terramon, & BLRMS (Off the top of the charts)
• H1 lost locked. L1 was already unlocked

   Locked for 58.75 hours. All looking good at this time. 

5.3 Magnitude EQ Triple Junction Region, Galapagos at 07:17utc, (00:17PT).

• Arrival ~07:45utc (00:45PT)
• Seen on USGS, Terramon, & BLRMS (0.2um/s at 0.03-0.1Hz),
• H1 & L1 remained locked.

TITLE: 05/01 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC

STATE of H1: Observing at 61Mpc

INCOMING OPERATOR: Jeff

SHIFT SUMMARY: Been locked for almost 55 hours. LLO joined us briefly but down again. Wind has pretty much died down. No issue to report.

FYI:

1. Note that the good news is I see H1 low latency hoft making it to CIT, and it is being aggregated. Also it seems the low-latency analysis can determine the lock segments from the hoft frames. 

Thus, the problems listed below are something we can investigate Monday morning. An email has been sent to DASWG and the relevant persons about this.

2. However, the redundant copy of hoft written by the DMT seems to have
stopped today at:

Apr 30 11:21 PDT.

Much earlier today, the DMT logs report Input/output error like this:

$ tail  logs/H1_rename_Frames-2017.04.25-09.43.log

‘/gds-h1/dmt/frames/hoft/H1/.temp/H-H1_DMT_C00-1177574268-4.gwf’ ->
‘/gds-h1/dmt/frames/hoft/H1//H-H1_DMT_C00-117757/H-H1_DMT_C00-1177574268-4.gwf’
mv: cannot move
‘/gds-h1/dmt/frames/hoft/H1/.temp/H-H1_DMT_C00-1177574268-4.gwf’ to
‘/gds-h1/dmt/frames/hoft/H1//H-H1_DMT_C00-117757/H-H1_DMT_C00-1177574268-4.gwf’:
Input/output error

3. Also, thus summary pages show dropout of the H1 lock segments

https://ldas-jobs.ligo-wa.caltech.edu/~detchar/summary/day/20170430/

and nagios reports errors like,

https://dashboard.ligo.org/cgi-bin/nagios3/status.cgi?host=all&servicestatustypes=28

"Known segments in frames that aren't in database"

Thus, low-latency data is making it to CIT, I think with the correct
locked segments, but these are not making into the segment database.

This is something we'll investigate Monday morning.

Evan G., Jeff K.

Summary:
The calibration measurement data that was collected yesterday has now been analyzed using our Markov Chain Monte-Carlo (MCMC) methods. We detail the results below. Nothing abnormal was found, and we find that the time varying factors can track changes in sensing gain and coupled cavity pole and actuation coefficients.

Details:
We analyzed the data collected during yesterday's calibration measurements, see LHO aLOG 35849. We have simplified the process for analyzing the data; rather than running two separate Matlab script to generate the MCMC results, we can now run just one script to get the final model results.

For the sensing function, we run
${CALSVN}/trunk/Runs/O2/H1/Scripts/SensingFunctionTFs/runSensingAnalysis_H1_O2.m
while for actuation, we run
${CALSVN}/trunk/Runs/O2/H1/Scripts/FullIFOActuatorTFs/analyzeActuationTFs.m.
Note that for the actuation calibration script, we have not yet converted it to an IFO agnostic script. Once it is converted, this script will be renamed.

Below are the table of values and their associated uncertainties for yesterday's measurements. Also, for comparison, are the modeled values from the reference measurement 3 Jan 2017:

                                                       Reference Value     MAP     (95% C.I.)        MAP       (95% C.I.)
                                                       2017-01-03          2017-01-03 (MCMC)         2017-04-27 (MCMC)
 Optical Gain                K_C         [ct/m]        1.088e6             1.088e6 (0.0002e6)        1.124e6   (0.0002e6)
 Couple Cav. Pole Freq.      f_c         [Hz]          360.0               360.0   (7.6)             343.4     (2.6)
 Residual Sensing Delay      tau_C       [us]          0.67                0.67    (6.7)             -1.8      (1.8)
 SRC Detuning Spring Freq.   f_s         [Hz]          6.91                6.91    (0.1)             7.4       (0.04)
 Inv. Spring Qual. Factor    1/Q_s       [ ]           0.0046              0.046   (0.016)           0.009256  (0.0069)

 UIM/L1 Actuation Strength   K_UIM       [N/ct]        8.091e-8            8.091e-8  (0.2%)          8.0818e-8 (0.18%)
 PUM/L2 Actuation Strength   K_PUM       [N/ct]        6.768e-10           6.768e-10 (0.02%)         6.795e-10 (0.08%)
 UIM/L3 Actuation Strength   K_TST       [N/ct]        4.357e-12           4.357e-12 (0.02%)         4.537e-12 (0.07%)
 UIM/L1 residual time delay              [usec]        n/a                 n/a                       29.1      (35.5)
 PUM/L2 residual time delay              [usec]        n/a                 n/a                       7.7       (3.1)
 TST/L3 residual time delay              [usec]        n/a                 n/a                       10.2      (1.8)


These values are derived from MCMC fitting to the data values. The attached plots show these results for the sensing and multiple-stage actuation functions.

We have added a new feature to the MCMC analysis, modeling the residual time delay in actuation. We expect to have zero usec of residual time delay, provided the model accurately captures all dynamics of the actuation. Deviations from zero can reveal un-modeled dynamics. For example, the PUM and TST residual time delays are inconsistent with zero usec, but we expect that this is due to imperfect modeling of the complicated violin resonances of the quad suspension.

The time varying factors are doing a good job tracking the changes between the reference model and the currently measured parameters (see time varying factors summary page).

For reference, the parameter files were used as follows:
${CALSVN}/trunk/Runs/O2/H1/params/2017-01-24/modelparams_H1_2017-01-24.conf    (rev4401, last changed 4396)
${CALSVN}/trunk/Runs/O2/H1/params/2017-04-27/measurements_2017-04-27_sensing.conf  (rev4596, last changed 4596)
${CALSVN}/trunk/Runs/O2/H1/params/2017-04-27/measurements_2017-04-27_ETMY_L1_actuator.conf  (rev4596, last changed 4596)
${CALSVN}/trunk/Runs/O2/H1/params/2017-04-27/measurements_2017-04-27_ETMY_L2_actuator.conf  (rev4596, last changed 4596)
${CALSVN}/trunk/Runs/O2/H1/params/2017-04-27/measurements_2017-04-27_ETMY_L3_actuator.conf  (rev4596, last changed 4596)