Evan, Dan, Sheila. Kiwamu

This afternoon we moved the picomotors on both transmon IR QPD sleds, although we moved Y much further than X.  This was to reduce the clipping and saturation Keita pointed out in 18077.  After that we excited TMSY in both pitch and yaw, and found error signals for the ITMs that are insensitive to TMS again. 

We have closed all the loops in full lock and things seem stable,  we were able to increase the power at 12:00 UTC on the 28th and saw that the recycling gain stayed stable.  We stopped at 6 Watts because the Y QPD is near saturation.  We should probably do more picoing (most of the light is on one quadrant), and reduce the whitening gain. 

The attached screenshot shows the settings we have now, the DSOFT and CSOFT loops could have gain of 1, but we were cautious. 

We had some difficulty turning on the ASC loops earlier tonight.  One thing that we noticed was that we were able to close INP1 each time, even when we didn't close PRC2 or CHARD (or SRC).  We also had trouble closing the CHARD PIT loop, although YAW was fine. Once we manualy adjusted the alignment to improve the recycling gain and closed the ITM loops, we closed CHARD PIT with no problems.  

We have left the ASC engage commented out in the guardian, because it is not working right now.  We also attempted earlier in the evening to take Keita's advice and manually aling the green PZT durring a full lock with good recycling gain.  We had difficulty getting the build ups high while minimizing the WFS signals, we adjusted the QPD offsets to keep the build ups high but this resulted in a bad recycling gain.  Probably a more thorough attempt is waranted. 

ETMX ESD tripping

Since saturday, the ETMX ESD has been tripping almost every lockloss.  We also need to toggle BO 4 to reset it, which is not required to reset ETMY.  

A follow up from the alog17973

A HWS operates at 1Hz at the corner station and 57Hz at the End stations. The first plot attached is a quiet DARM and PEM magnetometer reference during a full interferometer lock when none of the HWS cameras were on. The second plot shows spectra from the same channels when only the ITM HWS were on. It is difficult to determine the 1-Hz line due to high DARM noise floor while there is no clear evidence in the corner station magnetometer channel. The third plot shows the DARM spectrum and the PEM floor magnetometer when both ITM abd ETM HWS were on. A tiny peak just above the noisefloor appears in the EX magnetometer together with a peak in DARM. The choice of the chosen magnetometer axis was arbitary. The last three plots are the trend of HWS camera switches.

(This will not be reloaded into the working untill the morning)

As per request, I added a new 'OFFLINE' state to the IMC_LOCK Guardian.

This state will disengage the Common Mode Servo Board inputs (H1:IMC-REFL_SERVO_IN1EN and H1:IMC-REFL_SERVO_IN2EN) and misalign MC2.

Upon exiting this state, it will reengage the inputs and realign MC2 before heading through the DOWN state to aquire lock again, if desired.

Changes mostly related to our remapping of the ground STSs.  Both HEPI and ISIs for the BSCs are now getting their ground motion signal from the C (HAM5) STS2.  This required STS2CART matrix changes and changes in the match gains.

Additionally, Jim has disabled the Feed Forward on HAM4.  This will be the state upon restart.

There are SDF diffs on the End ISIs; these are the Stage1 Blends (Low vs High Wind.)  We will not restart these models.

safe.snaps commited to the svn.

J. Oberling, R. Savage, E. Merilh

Summary

After looking at the diode box chiller interlock signal H1:PSL-IL_DCHILFLOW and the actual flow sensor H1:PSL-OSC_DCHILFLOW we are starting to wonder if there might be a problem with the flow sensor.  It seems the interlock trips don't correlate to a drop in flow, but the other way around: an interlock trip triggers the drop in flow, which makes sense if the interlock is programmed to shut off the chiller in the case of an interlock trip, except this is not always the case and there doesn't seem to be a change in the flow rate before the interlock trips.  Details follow, investigation continues.

Details

In an effort to determine why the PSL tripped so many times over the weekend (see LHO alog 18063 and associated comments) we looked at the diode box chiller interlock signal H1:PSL-IL_DCHILFLOW and tried to correlate that to actual drops in the water flow of this chiller using the signal H1:PSL-OSC_DCHILFLOW.  Some examples from the weekend trips are attached below (I unfortunately forgot to save an overall picture of all 5 PSL trip events (6 diode box chiller flow interlock trips); will go back to the control room and attach as a comment).

PSL diode interlock trip event #1 - The interlock trips and after a few seconds either resets itself or someone was really on the ball.  The flow rate does not drop until several seconds after the initial trip (and after the interlock reset), and there is no change in the flow rate before the interlock trips.  Flow rate does not drop to zero but restores itself slowly.

PSL diode interlock trip event #2 -  2 interlock trips in quick succession, lasting just a few seconds each, once again with no change in the flow rate before the first interlock trip.  The flow rate of the chiller did not change until several seconds after the first initial trip; the second trip could be a result of the lower flow rate at the time of the trip.  Flow rate does not go to zero but restores itself (chiller shut off then back on?).

PSL diode interlock trip event #3 - One long trip for >8 minutes.  Hard to see on this plot but the interlock tripped a full 2 seconds before the flow actually stopped.  No change in flow rate before the interlock tripped.  Flow rate does go to zero this time and is restored shortly before the interlock resets.

PSL diode interlock trip event #6 - 3 interlock trips in short succession, each lasting just a few seconds.  Unlike the others, there is no associated drop in the chiller flow before or after the trip; the flow is being reported as within an acceptable range for operation with no obvious reason for the chiller flow interlock to trip.  Problem with the flow sensor?

Investigation ongoing, more to come.

700 - Cris, Karen to LVEA

1030 - Kyle to MX

1105 - Jeff B to Chiller Room

1109 - Jeff B out

1127 - Elli to LVEA to reposition some cameras

1148 - Elli out

1553 - Kyle back

Jim and Dave:

Prior to adding the RFM delayed write feature to H1 ALS tomorrow, we have tested the new feature on the DTS system over the past two days. We are using the h1alsex model running on x1iscex at 16kHz as the RFM IPC sender. The first test uses the h1fe3tim16 model running on x1lsc0 as the 16kHz receiver. Jim loaded up the filters on the sender until its CPU was in the 50uS range. With a 4km fiber run between sender and receiver (we go to the midstation and back) the receiver displayed 100% errors. Switching on the rfm_delay option on the sender zeroed the receiver errors.

The second test was to verify that the rfm_delay does no harm even if it is not needed. We installed zero filters on the sender, putting its CPU in the 27uS range. Delaying the signal caused no problems.

The third test was to check the data was not being corrupted (the receiver uses the cycle counter to check for errors). We put the received signal into the DAQ and sent a non-zero value over several hours. Trending the data shows no values received other than that sent.

The fourth test was to check a 16kHz sender and a 2kHz receiver (as is the case on H1 tomorrow with ALS sender and ASC receiver). We added a reciever to the h1fe3tim02 model, added it to the DAQ and trended its value and error channels. No errors were seen.

The rfm_delay has been tested and is ready for install tomorrow on H1.

restart logs for the previous week attached.

Summary:

Y ARM IR QPDB on TMS, which is used for CHARD and CSOFT and also for estimating recycling gain, is severly clipped. This makes both the ASC itself as well as the power recycling gain estimate fishy (LSC-TR_? comes from QPDB).

Fix it (or try fixing it) before using such a fishy signal for ASC.

Details:

Attached is the trend of one lock where Evan increased the power from 3W to 16W. The alignment of the cavity was changing during this due to power up operation.

If you see the IR QPD SUM on TMS (middle panel),  X_A, X_B and Y_A are all sort of proportional to each other, but Y_B is not, so it's apparent that it's clipping.

If you make B/A ratio for X arm it's dead flat, but that's definitely not the case for Y arm (top panel). The ratio for Y should be about the same as that of X, but Y ratio is always smaller and it gets worse as the power goes up. 

Also, at around t=370 sec, there's a kink in YB that is not related to the power kink. Though the clipping was definitely going on at t=300 and probably earlier, it seems like the clipping got worse at t=370.

Laser Status:
SysStat is good
Front End power is 31.4W (should be around 30 W)
FRONTEND WATCH is RED
HPO WATCH is RED

PMC:
It has been locked 0 day, 18 hr 49 minutes (should be days/weeks)
Reflected power is 2.2 Watts  and PowerSum = 24.4 Watts.
(Reflected Power should be <= 10% of PowerSum)

FSS:
It has been locked for 0 h and 32 min (should be days/weeks)
TPD[V] = 1.1V (min 0.9V)

ISS:
The diffracted power is around 8.6% (should be 5-9%)
Last saturation event was 0 h and 54 minutes ago (should be days/weeks)

Friday morning, the STS2-B (new) instrument was put under the igloo.  Attached areASDs, coherences and unit to unit TFs with original references--references from the old unit, within its igloo, legs still on vinyl.  The current traces are from 0130pdt 27 April.  The seismic environment was boring at that time.

Comparing these traces with the orignal (references) and the plots from Friday AM in aLog 18046:

The ITMY (STS2-B) Z DOF ASD looks much more like the other seismometers (STS2-A & C/HAM2 & HAM5) although the coherence with the A & C units has decreased in the 40-80 mHz region.

The X & Y DOFs don't show anything readily synthesized.  The unit to unit TFs show similar results as previous (aLog 18046) except the X DOF unity TF is now back up above 0.1 up from .07Hz on Friday.

The main thing I see is the improvement in the Z DOF (2nd attachment.)  All the units look very similar now in Z.

Maybe the long wavelength signals are predominantly vertical and the horizontals are a greater mix including short wavelengths...

SEI

• Prep for tomorrows HAM model restarts.
• Jim W will be taking measurements during that time.

CDS

• Gather AA/AI filters, Tue/Wed swapping
• Jim B to update software tools tomorrow

VAC

• Checking for leaks tomorrow morning at the ends.

PSL

• Investigating the 5 events from this weekend.
• Most likely need time to work on it tomorrow morning.

OpLev

• Hoping to have the ETMY OpLev up again by today.

Fac

• Water tank will be drained today
• Saturday there will be carpet cleaner. Make sure to take everyhing off of the floors, they will not be moving furniture.
• Next Mon there will be a painter on site.

Other

• Injection machine upgrade tomorrow
• This Fri/Sat is the Mini-Run, lets try to get some good data and prep for ER7

Building on Friday's work, I finished closing the ITM QPD loops. Settings are given in the attached screenshots. The pitch loops respond to a step at the error point with a 5 s time constant or so; the yaw loops respond with a 30 s time constant.

I installed 100000 ct limiters on the ITM ASC loop outputs (dSoft and cSoft), and 100 ct limiters on the TMS QPD A and QPD B pitch/yaw outputs. This prevents an infinitude of counts from being pushed onto the ITMs when the interferometer unlocks. [Failing to do this is punishable by 1 hour of roll mode damping.]

By adjusting the error point offsets, I was able to bring the recycling gain up to 38 W/W at 3 W of PSL power. I did not try super hard to push it up to 40 W/W this time.

I tried powering up to 6 W, but the recycling gain dropped to <30 W/W, the sideband buildups took a nosedive, and the ASC loops ran away. This unlocked the interferometer. I've seen this a couple times before (i.e., without the ITM loops). One story we could tell ourselves here is that the recycling gain became so low that the PR2 ASC error signals flipped sign and thereby made the loop unstable (my recollection is that the sign flip happens around a recycling gain of 30 W/W or so).

By trending the test mass oplevs (except ETMY's, which is not functional right now), one can see that most of the test mass motion during power up is usually in pitch. E.g., for a lock from a few days ago (2015-04-26 06:55:00 UTC, from 2.3 W to 11 W) I found IX moved by 0.61 µrad in pitch and 0.15 µrad in yaw, and EX moved by 0.20 µrad in pitch and -0.09 µrad in yaw.

I tried powering up a few times with the ITM loops off. The interferometer went from a recycling gain of 37 at 3 W to 35 at 10 W to 33 at 14 W. I tried moving the ITMs, PR3, and PRM in various combinations but could not seem to improve the recycling gain at 14 W.

I did not experience the 0.4 Hz instability today. The ITM oplev damping was on the whole time. Additionally, I tried powering up a few times with dc-coupled oplev loops feeding back to the ITM L1 stages (with the QPD loops off). This may have helped marginally in keeping the recycling gain from dropping too much, but it's hard to say.

Evan, Sheila,

Many things were tripped this morning after the earthquake in Nepal, (ISIs, HEPIs, and suspensions).  most of this went smoothly.  I had trouble untripping HAM5, the screen shot I attach is the collection of screens that I was looking at when I was confused, from these I thought that the watchdog was untripped, but that was untrue, in the ISI watchdog screen the small box aroud the rouge watchdog was red.  Maybe this could be more obvious, say if the watchdog indicator on the ISI overview screen was red when the rouge watchdog was tripped, and if the reset all button really reset all the watchdogs.  That might have save Jeff and Jim from a flurry of saturday morning texts in the future.  Jeff logged in and reset the rouge watchdog.  

Also, the PSL was tripped this morning about 16 UTC (not at the same time as the earthquake), it might have been a flow sensor problem.  After turning the laser on we weren't able to open the HPO external shutter from the beckhoff computer, we came to the control room and found that the flow sensor was red, we reset that but still weren't able to open the shutter from the control room.  We went back into the diode room and were able to reset the shutter from there.  

After the PSL came back SYS_DIAG said that the NPRO noise eater was oscillating, but we think that it is not.  The attached screenshot shows 100 days of the NPRO ROO readback, this number is around -500 when the noise eater is osciallating,  Each time that the laser tripps and is turnd on again this readback is high for a while before settling down.  We changed the tolerance on the readback to 50 counts from 10 counts in the SYS_DIAG test.