The watchdog indicator for the 70W amplifier was not correct. Now fixed. Both indicators correctly display the status of the power watchdogs now.
The glitches in the flow rate have gone since they were increased yesterday.
EX had a Dolphin glitch overnight. Peter is on site for a PSL issue, I'm starting the EX recovery remotely from home.
I gathered all log and proc files and then restarted all models on h1seiex, h1susex and h1iscex by running /etc/startWorld.sh.
I have reset all the software watchdogs on h1susex and h1seiex and cleared all DIAGs and CRCs
EX is now ready to be driven.
Time of this Dolphin crash (local time):
Nov 25 2018 05:50:18 PST
Would the first commissioner in the control room please call my cell phone so we can complete the recovery of EX.
A verbal alarm stating an earthquake was detected(?) just came in. The plot in the Control Room is about to cross the 1 um/s level. Things might be rung up for a while.
Opened FRS-11873
Dan is completing the EX recovery from the control room.
CS LVEA and Y-end VEA
I expect to be here for 1/2 hour or so and will make a comment to this entry when I leave.
Leaving site now.
Crystal chiller has just gone down again. Checked the fill tube and the water is below the minimum mark like yesterday.
Just saw Georgia Mansell's email from late last night about the chiller tripping again.
Just talked with Peter King. He and I will meet in the Chiller Room at 8:30 this morning to adjust chiller flows in hopes of curing these trips.
It appears that there are two issues.
First, it appears that the sensors cannot meter flows as low as we would like to maintain (1.2 lpm through the FrontEnd, 70-W, and High Power Shutter circuits).
Second, there is a small leak that the back of the chiller that was there since the flow throttling valve was installed about two weeks ago.
This was the system status screen as I found it this morning. Trend data says the flow from the crystal chiller fell precipitously about 10 hours ago.
PeterK and RickS
We filled the Crystal Chiller (about 500 ml) and started it. The flow displayed on the chiller controller was only about 13 lpm. We opened the throttle valve at the chiller output (fourth attached photo) to restore it to about 18 lpm (second attached photo).
Then, we adjusted the bypass valve on the wall (third attached photo) to give us about 40 PSI on the supply side (20 PSI on the return side) (first attached photo).
This results in about 1.7, 2.0, 1.7 (Front End, 70-W, HP shutter) lpm flows through the three Crystal Chiller circuits.
We also tightened the LXT (PVC) female adapter (to 3/4" NPT male street el) which appears to have stopped the leak at the chiller.
These flows may be larger than we will want for O3, but the system will hopefully run more reliably at these flow levels as we investigate options for reducing flows if required to reduce water-flow-induced noise on the PSL table.
No major issues since Tuesday's Dolphin driver upgrade. I did notice three things happened Wednesday evening between the hours of 19:00 and 20:25 PST: h1iopseih23 and h1iopseih45 had long'ish cpu times (13uS) just over an hour apart, and 20 minutes later some CRC errors appeared on h1seib3.
This afternoon I cleared the CRC and ran a full DIAG_RESET, soon after h1iopseih16 SWWD IPC receiver saw 5 errors in one second. As far as I am aware, this is the first time LHO has seen a transient SWWD IOP receive error, which has been seen at LLO previously. I'm not sure if its a coincidence that it appeared some seconds after the CRC/DIAG resets.
Daniel Brown, Cheryl Vorvick, Haocun Yu, Terry McRae, Georgia Mansell
Summary: Today we started recovering the corner IFO after the vent. We worked on IMC/IO revival, alignment, and squeezer alignment. The IMC recovered well and we have reasonable transmission and reflection levels.
We are currently having a guardian/front end problem when locking PRMI - The ISC_DRMI guardian tries to turn on H1:LSC-MICH1_FM2 when it reaches the PRMI_LOCKED state. This filter module has a yellow "LC" flag above it on the MEDM screen (does this mean it is under local control?), and toggling the button or trying to set it in a guardian shell does not work. We're not sure how to get around this. First attachment is the guardian log when trying to toggle FM2.
While PRMI acquires easily we're having trouble locking DRMI (a familiar situation with short locks but nothing that holds well).
General vent recovery
- I transitioned the LVEA to laser hazard
- Turned on the HAM6 PZT HV and fast shutter HV
- Turned on the fast shutter chassis (in the rack next to isct6), and enabled its high voltage
- note HAM6 HEPI is locked, so we left its guardian paused, in the READY state, and the SEI_HAM6 guardian in ISI_DAMPED_HEPI_OFFLINE
- We've set the gate_valve_flag parameter in lscparams to True, so we can do DRMI without arms
IMC revival
- We locked the IMC at 2 W, it was clearly misaligned in yaw to start off with. We adjusted MC2 so we could lock on the 00 mode nad used the WFS to fix the alignment
- Note to self, to get the WFS going from a pretty rubbish first alignment we had to lower the trigger threshold (from 40 to 21), then increase the IMC WFS gain by hand from 0.04 to 0.4. We waited for the WFS to converge then offloaded them with the IMC_LOCK guardian.
- The reflected power of the IMC is at 8.3 mW [edit: this was a typo, it's at 0.083 mW with ~1.9W input power], reduced again after two-tuesdays-ago's mystery excursion.
- Cheryl ran the IMC-spot-position measuring script. She tried to restore the IMC mirrors to their positions back in August, using the witness sensors, but found this was not a good alignment. She later found a good alignment.
Corner station alignment
- We mostly got through initial alignment with the ALIGN_IFO guardian without too many problems, we couldn't do input align without the arms locked though
- We first did MICH_DARK and MICH_BRIGHT to check the contrast defect with the new TCS settings. Initially we measured ~1% contrast defect, however after we tweaked up the input alignment this was reduced to ~0.65%.
- To adjust the input alignment we went to the PRC_ALIGN state of ALIGN_IFO and adjusted IM4 and PR2 by hand to increase LSC_POP_A_LF. The input alignment might not be optimal yet.
Squeezer work
We repeated the squeezer single-bounce tests reported in alog 45375, with more seed power. We have ~20 counts on AS_A and AS_B when bouncing the seed beam off the SRM, compared with 8 counts 1 week ago. We still didn't have enough power to see the squeezer beam reflected off ITMX when SRM is misaligned. The second attachment shows the squeezer seed beam, as reflected off SRM, visible on the AS_AIR camera on ISCT6. Qualitatively, it does not look obviously astigmatic.
Daniel Brown, Thomas Vo, Georgia Mansell
*actual channels I'm quoting are POPAIR_B_RF[18/90]_I_NORM_MON
Images of AS AIR on DRMI lock today with the current 50W pretunings.
We also began tuning CO2 differentially to see how the sideband build up on the OSA changes. The OSA scan attached is after the first step of 100mW change. Another 100mW change was applied but we lost DRMI and then the PSL went down.
Y-end VEA and CS LVEA
I expect to be here for < 1 hr. and will make a comment to this entry when I leave.
At some point (later today? tomorrow?) PT110 will change voltage ranges and will then be showing comparatively lower pressure values. After this occurs it will be helpful to revisit the state of the alarm texting system such that the Vacuum group receive alarm texts should PT100 or PT110 indicated pressures exceed X torr -> where "X" is the preset, traditional threshold pressure value (5 x 10-6 torr?).
0725 hrs. local -> Leaving site now
The nominal high alarm levels for PT100 (HAM1) and PT110 (HAM6) are 5.0e-05 Torr. Both of these gauges are currently in the 2.0e-06 Torr range. We can either wait until they drop below 5.0e-05 Torr or raise the alarm level to re-engage the cell phone alarms.
