TITLE: 09/01 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Lock Aquisition
INCOMING OPERATOR: Sep
SHIFT SUMMARY: Lots of commissioning, a PSL trip, and a small earthquake.
LOG:

• 01:59 PSL went out, Chiller error, flow sensor 1. Peter was awesome and drove out to the site to get it back on and going for more commissioning.
• 03:14 Dave restarting H1SUSPROCPI

the h1susprocpi model had two entries in the testpoint.par file (dcuids 52 and 71). The correct ID is 71. This was causing a failure to open testpoints on this model. After the testpoint.par file was corrected, I restarted h1susprocpi and testpoints are now available.

Attached are trends of the various flow sensors in the PSL.  Comparison of the timestamps shows that the high power oscillator
tripped because of a flow rate problem with the AMP circuit.  The problem does not lie within the 4 laser heads, leaving either
the front end power amplifier, or the 4 crystal chambers.

    Unfortunately checking the 4 crystal chambers requires dismantling the laser.  The vortex flow sensor in the AMP circuit
is relatively new.  The other object in that circuit is the power amplifier module.  An inspection mirror might be able to
afford a view of the plumbing underneath the housing.

    One can see that the flow rate in the AMP circuit drops before the output of the front end laser drops, and that the output
power of the front end laser drops before the high power oscillator.  I think the sequence of events is the following:
 1. flow rate problem in the AMP circuit trips the flow watch dog of the front end laser
 2. the switching off of the front end laser breaks the injection locking of the high power oscillator
 3. the loss of injection locking results in a power drop in the high power oscillator which then trips the power watch dog
 4. the power watch dog switches the high power oscillator off

WP6121: Jonathan, Carlos, Jim, Dave

Tuesday Aug 30, the Remote Access Control system (RACCESS) was turned on during the times we have operator coverage (Mon-Fri, 8am - 4pm Pacific). This is a 'real life' test of the system, and an opportunity for everyone to get familiar with it before it is on full-time during O2. Any problems with the system should be communicated as an LHO-CDS FRS ticket.

To provide more visibility of who is logged into the border machine (cdsssh) I have expanded the RACCESS portion of the CDS_OVERVIEW medm (center right area).

WP 6131. Jonathan's new daqd code which is running on both h1fw0 and h1fw1 exports more signals via EPICS channels. At 12:42PDT today I restarted the DAQ with a new H1EDCU_DAQ.ini file which includes the full set of EPICS channels. I also added the standard set of channels for the h1fw2 frame writer.

I have extended the DAQ Overview medm to show trend file sizes, and added links to open the detailed screens for fw0 and fw1 which show the full diagnostic suite.

I redid our green initial alignment setpoints, in hopes that CHARD won't have such a large input offset when we try to engage the loops.

Over-filled CP3 with the exhaust bypass valve fully open and the LLCV bypass valve 1/2 turn open.

Flow was noted after 23 minutes 15 seconds, closed LLCV valve, and 3 minutes later the exhaust bypass valve was closed.

I raised CP3's LLCV from 19% to 20%, due to the time increase for the last two fills.

TITLE: 08/31 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
INCOMING OPERATOR: TJ
SHIFT SUMMARY:  First half of today was dedicated to Kiwamu's SRC Gouy phase measurements.  After lunch, I did an IA, and after some troubleshooting with Sheila and Jenne of ALS Diff, we are back to locking.
LOG:

16:20 Kissel taking TMSx TFs, Kiwamu starting SRC measurements

16:24 Keita to LVEA making ISS 2nd loop electronics measurements

16:39 Kiwamu to LVEA byu ISCT6 for SRC Gouy meas.

16:44 Karen done at MY

16:48 Cristina done at MX

17:05 Betsy, John, and Chandra to MY

17:35 Betsy, John, and Chandra to MX

18:03 Kissel done with TMSx

19:31 Kiwamu done

19:34 Keita to LVEA for more 2nd loop meas.

20:29 Chandra to LVEA

20:43 Chandra done

21:21 Gerardo to MY

22:22 Gerardo done

22:56 Jim switching ISIs to Earthquake mode for incoming EQ

 

Only HAM2 required a WD counter reset.  See attached screenshot.

The CO2 heating power was calculated based on Aidan's CO2 power vs. PSL power plot (alog25932). With the new thermal lensing measurement (alog28799) I fine-tuned the equation.

CO2 power = slope * PSL power + offset

ITMX slope was -0.01, now -0.012

ITMY slope was -0.01, now -0.014

ITMX offset was 0.5, now  0.6

ITMY offset was 0.3, now 0.4

Elli (remotely), Kiwamu,

This morning was a morning for another measurement of the SRC gouy phase. As opposed to the single bounce measurements done yesterday, we proceeded to a round trip beam measurement.

I wanted to measure two different configurations as instructed by Elli, but I could get only one of them done today. We will spend (at least) another morning to measure the other configuration.

[Some background]

A trick in this whole series of measurements is that one can effectively cancel the effect of the output optical train (i.e. the optical path from SRM all the way to the setup on ISCT6 which is not easy to precisely characterize)  by having measurements of single bounce and round trip beams. The round trip beam we mean here is a beam that bounces around the signal recycling cavity only once and comes out to the AS port. We have finished the single bounce measurement yesterday, and therefore the next step today was to measure the round trip beam.

[The setup]

I added an extra component to ISCT6. It is a beam blocking object. Everything else was unchanged on ISCT6. See the picture of the setup shown below.

The blocking object (a black rectangular piece in the middle) is dedicated to block the single bounce beam which would cause undesired interference with the round trip beam. To split the beam into single and round trip beams, I introduced an intentional misalignment in SRM by 700 urad in pitch as suggested by Elli. In addition, I found that the separation of the two beams became even better with additional misalignment (~ 100 urad) in pitch of PR2 as well. This misaligned configuration is one of two configurations we wanted to test. The other configuration will introduce misalignment in another combination of optics, ITM and IM4, instead of SRM.

As I introduced misalignment in SRM, it made a clean beam separation on CAM17 (see the previous log) while ASAIR did not as expected. I manually steered a mirror and beam splitter that were in front of the cameras to center the round trip beam on both cameras. The blocking object was removed when I finished the measurement this morning.

- - - - some other settings.

PSL power into IMC = 25 W

ITMY ring heater = 0.5 W (0.25 W for upper and lower segments each)

CO2Y = 286 mW

The interferometer configuration = single bounce (with ITMY aligned) + SRM almost aligned (see the second and third attachment for the specific alignment values)

The camera settings = same as the previous measurements (alog 29389)

ASAIR exposure time = 4400 usec

CAM17 exposure time = 7000 usec

[The measurement]

The measurement itself is the same as what we did yesterday -- excite BS or PR2 in yaw at a certain frequency and measure the centroid positions on the two gigE cameras. I ended up doing four sets of measurements as described below because I was worried that a high excitation may have introduced a large enough clipping somewhere which may confuse the later analysis. By the way, later Jenne told me that there were some angular excitation signals unintentionally left on throughout the measurements on BS and all the SR mirrors (in both pitch and yaw) at frequencies around 20 Hz, which I don't think an issue because they are small compared to my measurement excitation and also the frequencies are different than my excitation.

- measurement #1

 18:06:40 - 18:16:40 UTC

 BS yaw excitation by 6 urad at 0.2 Hz (ASAIR camera showed a clipping-type behavior)

- measurement #2

 18:19:53 - 18:29:53 UTC

 BS yaw excitation by 3 urad at 0.2 Hz

- measurement #3

 18:34:15 - 18:44:15 UTC

 PR2 yaw excitation by 20 urad at 0.2 Hz (ASAIR camera showed a clipping-type behavior)

- measurement #4

 18:47:15 - 18:57:15 UTC

 PR2 yaw excitation by 10 urad at 0.2 Hz

[The data]

A thorough analysis will be remotely performed by Elli. The data are saved in kiwamu.izumi/Public/measurements/20160831_SRCgouy2/data1

J. Kissel

While perusing the list of FRS tickets / Integration issues that had been opened and related to me, I found an old issue -- now FRS #3246 -- that cites LHO aLOG 19208. The story from that aLOG : after the vent in the summer of 2015, the LF & RT OSEM signal chains had shown a factor of a few less gain than their values prior to the vent.

I was tempted to close the issue, claiming the cop out "well, we detected gravitational waves with TMSX like this..." but there happened to be some time this morning, where then end stations were free. Also, given the typical-forgetten-about state of the TMTS, this transfer function had not been remeasured since just after the pump-down of the chamber -- not at final vacuum levels.

As such, I've remeasured the standard top-to-top transfer function of H1SUSTMSX, and found even further a drop in transfer function magnitude. The transfer functions confirm that V and P -- transfer functions show a factor of 4 drop in response from the 2014 to 2016 measurements. Additionally, and not mentioned in the original fault (although present in the 2015 transfer functions) report R also shows a drop from 2014 to 2016 of a factor 2. See first pdf  attachment (alltmtss_2016-08-31_Phase3b_H1SUSTMSY_M1_ALL_ZOOMED_TFs.pdf).

The drop in plant gain leaves the R, P and especially V DOFs with little to no damping on resonance.

Digging even further, recall that TMTS top masses (M1) are rotated 90 deg to that of the QUAD, so the DOF mapping (for the relevant DOFs in question) is 
    V --> LF and RT
    R --> F1, F2, and F3
    P --> LF and RT
Since L (= SD), T, and Y (= F2 and F3) look the same, we can rule out problems with F2, F3, and SD. This leaves LF, RT, and F1 as our suspect OSEMs.

Unlike suspected before, I'm not sure if this is an external electronics chain issue. Why? Because, typically electronics chain problem show up clustered in an entire satellite amplifier or coil driver. The TMTS OSEMs are group in the typical six-osem stage fashion of F1, F2, F3, LF on one cable chain, and RT, SD on another (see pg 3 of D1002741).

I've also attached some new figures (which are standard output of the transfer function scripts, but posting them had fallen out of fashion) that compares the response in the OSEM basis to Euler basis drive. This isolates the individual sensor composition of each DOF. See the rest of the .pdf attachments (
H1SUSTMSX_M1_*.pdf), which compare the 2014 and 2016 data sets in this manner. This shows a consistent story, that
    - F1 alone (as opposed to F2 and F3) has dropped in sensitivity by a factor of 2. 
    - LF has dropped in sensitivity by a factor 6, and RT has dropped by a factor of 3.

I then went on to wonder -- seeing the trend from 2014 to 2016 -- have the OSEM LEDs just slowly decayed in sensitivity over time? This launched the data viewer mining exercise for all of the attached .pngs, H1SUS${OPTIC}_${TOPSTAGE}_OSEMINF_3yr_Trend.png. These are hourly trends of the mean value for each OSEM over the past three years. I was hoping to see the H1SUSTMSX's F1, LF, and RT OSEMs showed a slow, but substantial, downward decay in raw input ADC voltage over time, with the hypothesis that this represented a slow failure of those OSEM's LEDs or PDs. Sadly, evidence from other suspensions I checked showed that a random smattering of BOSEMs scattered around all BSC SUS show either flat or a slow decay of at most ~3000 [ct]; the rest are flat in time. This drop in PD current is only about 10% of the full range (~30000 [ct]), so it cannot explain the factors of 2 to 6. 

In conclusion, I've recommended that we close this ticket with the LONGTERMFIX, and open up a proper Integration Issue about it, marking it WHEN VENT, because I have a feeling this will be easier to debug when we have access to the entire signal chain. In the mean time, we can band-aid the problem by increasing the overall gain the V, R, and P damping loops by the amount of plant sensitivity that has been lost.

Unsuccessful at damping ITMX 15520 Hz PI several times tonight (previously seen here and here). We find that larger damping drive does not equal greater damping: When this mode was test driven and damped after the thermal transient at 50 W, a best gain and phase was found for damping. When the mode began to ring up later, increasing gain (by some large amound but still under saturation) or flipping sign and/or changing phase only resulted in faster ringup. This is true when still far below DAC saturation levels. It seems as if there is some gain sweet spot that must be found. 

--- --- ---

We had three occasions to damp ITMX 15520 Hz mode during the night. During the first, I successfully damped and it then rerang up (perhaps due to offset adjustments?) and lost lock. During the second and third I was not able to damp the mode and avoided lockloss only by decreasing power 50 W --> 25 W.

Below you see the mode first ring up and my gain trial and error response until I settle at 10000 and the mode is fully damped. Soon after, you see the mode ring up again right after the yaw offset step from ~ -0.02 --> -0.01. Note we started with a small negative gain so I just assumed we had actually been ringing up the mode the past few days.

During the second 50 W lock, damping was already running at the gain and phase settings that were effective at damping the first ring up above (+10000 gain, -60deg). Despite this, you see the mode slowly rising ~3 hours into the 50 W lock and my gain adjustments trying to damp. Note that here I start with some mostly successful positive gain (i.e. shallow slope) yet both raising the gain and flipping the gain sign cause the mode to ring up. I tried phase changes at this point too but existing was best. I avoided lockloss by decreasing power to 25 W, allowing mode to ring down enough to damp, then powering back up. I also rechecked my gain and phase at 50 W (post thermal transient time) and it would still drive and damp with a very steep slope. Still, an hourish later the mode began to ring up. I found similar behavior in the third attempt as the second and had to decrease power to avoid lockloss. 

Things to note: 

• After full lock aquisition, this mode takes ~ 3 hours to ring up. This morning, after dropping to 25 W then coming back to 50 W, it took only ~ 1.5 hours to ring up. Suggests something thermal.
• Q_m of ITMX ~= 18M
• Gap between ESD and ITM is larger than for ETMs ==> less actuation per drive
• Kiwamu has found similar higher gain =/= better damping in violin mode damping

Things to try:

• Retry damping immediately after decreasing power; should be able to see within ~1 min of lower power. 
• Try driving only one quadrant; test spatial overlap of ESD and mechanical mode
• Turn up ring heater (most likely ETMX RH) to decrease frequency overlap between ITMX mech mode and optical mode
• Check offset dependence

Added 100 mL H2O to crystal chiller.

This completes FAMIS task 6486.

6.7 in Papua New Guinea shook us about for a few hours. We put the SEI configuration to EarthQuake and then tried locking after a handful of minutes and it looked like it was doing well, getting us to SWITCH_TO_QPDS before losing lock. When the seismometers looked low enough and we still could not lock, we tried switching back to WINDY and then we immediately noticed a difference and made it all the way to DC_READOUT.

Perhaps the EarthQuake configuration is not as good as we previously thought.

Jeff K, Chris Whittle

We used the interometer uptime to test the ITM charge measurement script by taking data for ITMX. With the previously used excitation frequency of 20.1 Hz being used for other measurements, we opted for an excitation at 11.5 Hz. At this frequency, we found an excitation amplitude of 1k counts to be sufficient for a good SNR. We have taken data at bias voltages between -10k and 10k counts, but have yet to perform post-processing.

We additionally tested whether flipping the analog voltage out switches to the quadrants had any effect on the locked interferometer. No effects were observed for flipping on/off quadrant voltages in various orders. This test was performed on just ITMX and with no actual output voltage to the quadrants.

J. Kissel

In short: The ETM ESD bias signs have now been flipped.

After whining about it since ER9, I've commissioned why the bias sign flipping had caused the ALS DIFF control to go unstable (and subsequently DARM once we get onto ETMY): controlling the loop gain sign beyond the ESD linearization just doesn't work. As such, I've restored all settings for both test masses to their successful settings back in April -- namely that from LHO aLOG 26826. As such, we've returned to the aesthetically displeasing but functional method of controlling the DARM loop sign in the DRIVEALIGN matrix. 

We still have yet to assess the impact on PI damping.

----------------------

Explicit details for the next time this becomes confusing:
ETMX
(1) Changed H1:SUS-ETMX_L3_LOCK_INBIAS from -9.5 [V] to +9.5 [V]
(2) Changed 
       H1:SUS-ETMX_L3_DRIVEALIGN_L2L_GAIN from +1.0 to -1.0
       H1:SUS-ETMX_L3_DRIVEALIGN_L2P_GAIN from +0.021 to -0.021 
       H1:SUS-ETMX_L3_DRIVEALIGN_L2Y_GAIN from +0.007 to -0.007
(3) Changed H1:SUS-ETMX_L3_ESDOUTF_LIN_FORCE_COEFF from -124518.4 to +124518.4
(4) Made sure all H1:SUS-ETMX_L3_ESDOUTF_??_GAIN fields are +1 always, all the time, as before.

(No changes need to the calibration model since we don't use ETMX in our lowest noise state.)

ETMY
(5) Changed H1:SUS-ETMY_L3_LOCK_INBIAS from +9.5 [V] to -9.5 [V]
(6) Changed H1:SUS-ETMY_L3_DRIVEALIGN_L2L_GAIN from +30.0 [V] to -30.0 [V]
(7) Changed H1:SUS-ETMY_L3_ESDOUTF_LIN_FORCE_COEFF from +124518.4 to -124518.4 (even though ETMY doesn't use linearization).
(8) Made sure all H1:SUS-ETMX_L3_ESDOUTF_??_GAIN fields are +1 always, all the time, as before.

(9) Changed H1:CAL-CS_DARM_FE_ETMY_L3_DRIVEALIGN_L2L_GAIN from +30 to -30
(10) Changed H1:CAL-CS_DARM_FE_ETMY_L3_ESDOUTF_UL_GAIN from -1 to +1, where it should remain always, all the time, as before.
(11) Changed H1:CAL-CS_DARM_FE_ETMY_L3_ESDOUTF_LIN_FORCE_COEFF from +124518.4 to -124518.4 (even though ETMY doesn't use linearization).

I've also redone (essentially reverted) the DOWN state in the ISC_LOCK guardian with respect to the ETM ESD settings, such that steps 2-3, and 6-11 are done automatically if a user does steps 1 and 5. Once we figure out the impact on PI damping, we'll code these up in the DOWN state of the ISC_LOCK guardian as well. 

Finally, I've accepted these changes into the 
H1SUSETMX down.snap (to which its safe.snap is a soft link)
H1SUSETMY down.snap (to which its safe.snap is a soft link)
H1CALCS   safe.snap and OBSERVE.snap.
SDF systems.

This morning at about 16:00UTC I made some adjustment to ETMY oplev as per WP #6123. I've included some photos of a 5day trend as well as a before and after 5 the AA reset. THe output power was increased by 7mV in an attempt to stop glitches. I don't know if the after reset pic is of any use as the suspension hadn't quite settled down.

PSL Team currently investigating the lastest tripping of the LASER. The trends don't really show any strangeness until AFTER the interlock trip. 29356

the Verbal Alarms code was logging to the ops home directory. Prior to the move of this home directory (WP5658) I have modified the code to log to a new directory:

/ligo/logs/VerbalAlarms

We restarted the program at 14:04 and verified the log files are logging correctly.