Work Permit 5173 Today we swapped both EX and EY ISC AA AI chassis with V6 AA/AI chassis. There were 2 AA and 1 AI at each end station. We also verified the PEM and PCal were already at V6. Tomorrow we will continue with SUS and SEI.
WP #5164, 5170 Installed version branch/gds-2.16.17.2 GDS tools for Ubuntu, which includes a bug fix for printing problems in diaggui and foton. Adds an option to foton for use with non-aLIGO filter development. Adds enhancements to diaggui to search for awgtpman servers on multiple networks, and to restrict the search for awgtpmans to specific models (not for general use). Installed minor updates to dataviewer and command line nds tools to recognize NDS protocol version 12.2, which was installed with the RCG 2.9.1 update last week. Should be no functional change to those tools.
H. Radkins, J. Kissel, Sadly we had to break the very-long-standing RF lock. I've brought all IFO / ISC guardians to DOWN, and I've reloaded the IMC guardian in hope to receive T.J.'s new "OFFLINE" state for the IMC, but there's some bug there. I've left it in INIT and misaligned MC2. Hugh is beginning to bring the platforms to OFFLINE.
Evan, Dan, Sheila. Kiwamu
This afternoon we moved the picomotors on both transmon IR QPD sleds, although we moved Y much further than X. This was to reduce the clipping and saturation Keita pointed out in 18077. After that we excited TMSY in both pitch and yaw, and found error signals for the ITMs that are insensitive to TMS again.
We have closed all the loops in full lock and things seem stable, we were able to increase the power at 12:00 UTC on the 28th and saw that the recycling gain stayed stable. We stopped at 6 Watts because the Y QPD is near saturation. We should probably do more picoing (most of the light is on one quadrant), and reduce the whitening gain.
The attached screenshot shows the settings we have now, the DSOFT and CSOFT loops could have gain of 1, but we were cautious.
We had some difficulty turning on the ASC loops earlier tonight. One thing that we noticed was that we were able to close INP1 each time, even when we didn't close PRC2 or CHARD (or SRC). We also had trouble closing the CHARD PIT loop, although YAW was fine. Once we manualy adjusted the alignment to improve the recycling gain and closed the ITM loops, we closed CHARD PIT with no problems.
We have left the ASC engage commented out in the guardian, because it is not working right now. We also attempted earlier in the evening to take Keita's advice and manually aling the green PZT durring a full lock with good recycling gain. We had difficulty getting the build ups high while minimizing the WFS signals, we adjusted the QPD offsets to keep the build ups high but this resulted in a bad recycling gain. Probably a more thorough attempt is waranted.
ETMX ESD tripping
Since saturday, the ETMX ESD has been tripping almost every lockloss. We also need to toggle BO 4 to reset it, which is not required to reset ETMY.
durring this 2.25 hour lock, the recycling gain stayed fairly stable. As the POP LF trend in the attached screenshot shows, the recylcing gain dropped by about 5% in the first half hour of the lock. the Y trans mon QPDs see a drift in yaw on a similar timescale, although the X arm yaw loop control signal (DSOFT Y OUT) doesn't respond to this, which would indicate that the drift is from TMS, which our combination of error signals is chosen to be insensitive to. In conclusion it seems like the QPD loops are doing their job. As keita suggested, it is possible that the change in recycling gain in the first half hour is a thermal effect.
A follow up from the alog17973
A HWS operates at 1Hz at the corner station and 57Hz at the End stations. The first plot attached is a quiet DARM and PEM magnetometer reference during a full interferometer lock when none of the HWS cameras were on. The second plot shows spectra from the same channels when only the ITM HWS were on. It is difficult to determine the 1-Hz line due to high DARM noise floor while there is no clear evidence in the corner station magnetometer channel. The third plot shows the DARM spectrum and the PEM floor magnetometer when both ITM abd ETM HWS were on. A tiny peak just above the noisefloor appears in the EX magnetometer together with a peak in DARM. The choice of the chosen magnetometer axis was arbitary. The last three plots are the trend of HWS camera switches.
(This will not be reloaded into the working untill the morning)
As per request, I added a new 'OFFLINE' state to the IMC_LOCK Guardian.
This state will disengage the Common Mode Servo Board inputs (H1:IMC-REFL_SERVO_IN1EN and H1:IMC-REFL_SERVO_IN2EN) and misalign MC2.
Upon exiting this state, it will reengage the inputs and realign MC2 before heading through the DOWN state to aquire lock again, if desired.
New graph attached
I'm looking at the code changes here (which I had to do by logging into the site, since they haven't been committed to the SVN) and I see some issues that will likely prevent it from working as intended.
TJ: I suggest you talk to me before you try to deploy this.
Changes mostly related to our remapping of the ground STSs. Both HEPI and ISIs for the BSCs are now getting their ground motion signal from the C (HAM5) STS2. This required STS2CART matrix changes and changes in the match gains.
Additionally, Jim has disabled the Feed Forward on HAM4. This will be the state upon restart.
There are SDF diffs on the End ISIs; these are the Stage1 Blends (Low vs High Wind.) We will not restart these models.
safe.snaps commited to the svn.
J. Oberling, R. Savage, E. Merilh
Summary
After looking at the diode box chiller interlock signal H1:PSL-IL_DCHILFLOW and the actual flow sensor H1:PSL-OSC_DCHILFLOW we are starting to wonder if there might be a problem with the flow sensor. It seems the interlock trips don't correlate to a drop in flow, but the other way around: an interlock trip triggers the drop in flow, which makes sense if the interlock is programmed to shut off the chiller in the case of an interlock trip, except this is not always the case and there doesn't seem to be a change in the flow rate before the interlock trips. Details follow, investigation continues.
Details
In an effort to determine why the PSL tripped so many times over the weekend (see LHO alog 18063 and associated comments) we looked at the diode box chiller interlock signal H1:PSL-IL_DCHILFLOW and tried to correlate that to actual drops in the water flow of this chiller using the signal H1:PSL-OSC_DCHILFLOW. Some examples from the weekend trips are attached below (I unfortunately forgot to save an overall picture of all 5 PSL trip events (6 diode box chiller flow interlock trips); will go back to the control room and attach as a comment).
PSL diode interlock trip event #1 - The interlock trips and after a few seconds either resets itself or someone was really on the ball. The flow rate does not drop until several seconds after the initial trip (and after the interlock reset), and there is no change in the flow rate before the interlock trips. Flow rate does not drop to zero but restores itself slowly.
PSL diode interlock trip event #2 - 2 interlock trips in quick succession, lasting just a few seconds each, once again with no change in the flow rate before the first interlock trip. The flow rate of the chiller did not change until several seconds after the first initial trip; the second trip could be a result of the lower flow rate at the time of the trip. Flow rate does not go to zero but restores itself (chiller shut off then back on?).
PSL diode interlock trip event #3 - One long trip for >8 minutes. Hard to see on this plot but the interlock tripped a full 2 seconds before the flow actually stopped. No change in flow rate before the interlock tripped. Flow rate does go to zero this time and is restored shortly before the interlock resets.
PSL diode interlock trip event #6 - 3 interlock trips in short succession, each lasting just a few seconds. Unlike the others, there is no associated drop in the chiller flow before or after the trip; the flow is being reported as within an acceptable range for operation with no obvious reason for the chiller flow interlock to trip. Problem with the flow sensor?
Investigation ongoing, more to come.
Attached is an overview of all the trips of H1:PSL-IL_DCHILFLOW from this last weekend. I also included the general PSL interlock, H1:PSL-IL_OK, and the NPRO OK signal H1:PSL-AMP_NPROOK. As can be seen the diode chiller flow interlock tripped 6 times, causing 5 trips of the general PSL interlock, and 5 corresponding drops of the NPRO (2 trips of the chiller interlock, #s 4 and 5, occured before the general PSL interlock was reset). As previously noted something is causing the diode chiller flow interlock to trip, and it doesn't seem to be the diode chiller flow. Investigation continues...
700 - Cris, Karen to LVEA
1030 - Kyle to MX
1105 - Jeff B to Chiller Room
1109 - Jeff B out
1127 - Elli to LVEA to reposition some cameras
1148 - Elli out
1553 - Kyle back
Jim and Dave:
Prior to adding the RFM delayed write feature to H1 ALS tomorrow, we have tested the new feature on the DTS system over the past two days. We are using the h1alsex model running on x1iscex at 16kHz as the RFM IPC sender. The first test uses the h1fe3tim16 model running on x1lsc0 as the 16kHz receiver. Jim loaded up the filters on the sender until its CPU was in the 50uS range. With a 4km fiber run between sender and receiver (we go to the midstation and back) the receiver displayed 100% errors. Switching on the rfm_delay option on the sender zeroed the receiver errors.
The second test was to verify that the rfm_delay does no harm even if it is not needed. We installed zero filters on the sender, putting its CPU in the 27uS range. Delaying the signal caused no problems.
The third test was to check the data was not being corrupted (the receiver uses the cycle counter to check for errors). We put the received signal into the DAQ and sent a non-zero value over several hours. Trending the data shows no values received other than that sent.
The fourth test was to check a 16kHz sender and a 2kHz receiver (as is the case on H1 tomorrow with ALS sender and ASC receiver). We added a reciever to the h1fe3tim02 model, added it to the DAQ and trended its value and error channels. No errors were seen.
The rfm_delay has been tested and is ready for install tomorrow on H1.
restart logs for the previous week attached.
Summary:
Y ARM IR QPDB on TMS, which is used for CHARD and CSOFT and also for estimating recycling gain, is severly clipped. This makes both the ASC itself as well as the power recycling gain estimate fishy (LSC-TR_? comes from QPDB).
Fix it (or try fixing it) before using such a fishy signal for ASC.
Details:
Attached is the trend of one lock where Evan increased the power from 3W to 16W. The alignment of the cavity was changing during this due to power up operation.
If you see the IR QPD SUM on TMS (middle panel), X_A, X_B and Y_A are all sort of proportional to each other, but Y_B is not, so it's apparent that it's clipping.
If you make B/A ratio for X arm it's dead flat, but that's definitely not the case for Y arm (top panel). The ratio for Y should be about the same as that of X, but Y ratio is always smaller and it gets worse as the power goes up.
Also, at around t=370 sec, there's a kink in YB that is not related to the power kink. Though the clipping was definitely going on at t=300 and probably earlier, it seems like the clipping got worse at t=370.
Evan found that Y QPDB was railing in high power, and it turns out that the odd kink was indeed when the Y QPDB segment 4 started railing.
Still, the departure from the flat line right after the power started increasing from 2.4W indicates that there is a clipping.
Laser Status:
SysStat is good
Front End power is 31.4W (should be around 30 W)
FRONTEND WATCH is RED
HPO WATCH is RED
PMC:
It has been locked 0 day, 18 hr 49 minutes (should be days/weeks)
Reflected power is 2.2 Watts and PowerSum = 24.4 Watts.
(Reflected Power should be <= 10% of PowerSum)
FSS:
It has been locked for 0 h and 32 min (should be days/weeks)
TPD[V] = 1.1V (min 0.9V)
ISS:
The diffracted power is around 8.6% (should be 5-9%)
Last saturation event was 0 h and 54 minutes ago (should be days/weeks)
Friday morning, the STS2-B (new) instrument was put under the igloo. Attached areASDs, coherences and unit to unit TFs with original references--references from the old unit, within its igloo, legs still on vinyl. The current traces are from 0130pdt 27 April. The seismic environment was boring at that time.
Comparing these traces with the orignal (references) and the plots from Friday AM in aLog 18046:
The ITMY (STS2-B) Z DOF ASD looks much more like the other seismometers (STS2-A & C/HAM2 & HAM5) although the coherence with the A & C units has decreased in the 40-80 mHz region.
The X & Y DOFs don't show anything readily synthesized. The unit to unit TFs show similar results as previous (aLog 18046) except the X DOF unity TF is now back up above 0.1 up from .07Hz on Friday.
The main thing I see is the improvement in the Z DOF (2nd attachment.) All the units look very similar now in Z.
Maybe the long wavelength signals are predominantly vertical and the horizontals are a greater mix including short wavelengths...
SEI
CDS
VAC
PSL
OpLev
Fac
Other
Additional CDS work to Tuesday maintenance: upgrade end station ALS models to RCG branch2.9, use new RFM_DELAY feature for all ALS RFM senders to remove ASC receive errors.
Correction on injection work, no upgrade of the hardware injection machine, new h1calcs model to be installed.
IMC_LOCK has been fixed and works now. Hopefully it stays that way. Things changed are:
This has now been committed to the svn.