Had to accept 2 SDF diffs for CS ECAT PLC2 to go to Observing. See screenshot.
PI modes 27 and 28 rang up while I was out of the CR.
Lockloss as a few smallish, magnitude ~5 EQs were ringing down. EQ band on BLRMS was right at 10^-1 um/s, so it doesn't seem like this should have been the cause. Otherwise, nothing ringing up unusually.
TITLE: 12/18 Owl Shift: 08:00-16:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 71.0922Mpc
OUTGOING OPERATOR: Cheryl
CURRENT ENVIRONMENT:
Wind: 8mph Gusts, 7mph 5min avg
Primary useism: 0.08 μm/s
Secondary useism: 0.38 μm/s
QUICK SUMMARY: Walked in to a locked IFO. Right as my shift was starting, we were dropped out of Observing due to TCS_ITMY_CO2 Guardian node. It said it was trying to lock the laser. As I was starting to investigate the cause, it fixed itself. We are now back in Observing.
Also, since we were already out of Observing and LLO is down, I took the opportunity to run A2L.
State of H1:
| Reported by | Magnitude | Location | |
| Terramon | yes |
5.1 |
-34.9, LON: -107.8 |
| USGS | yes | 5.1 | 34.915°S 107.795°W |
| SEISMON | no | - |
|
|
Reported by | Magnitude | Location |
| Terramon | yes | 5.8 |
LAT: -6.3, LON: 154.3 |
| USGS | yes | 5.8 | 6.332°S 154.337°E |
| SEISMON | no | - |
| Reported by | Magnitude | Location | |
| Terramon | yes | 5.1 |
LAT: -6.4, LON: 154.5 |
| USGS | yes | 5.1 | 6.436°S 154.512°E |
| SEISMON | no | - |
State of H1: unlocked, recovering
Activities:
Update: as of 2:08UTC, ALS X arm is locked at 0.92, COMMA_DEMOD is at 3.7 which is low, X arm in IR is flashing at 0.7, I'm declaring relative victory and moving on to the Y arm
X arm recovery:
Update: as of 2:41UTC, ALS Y arm locked at 1.05, X arm needs some attention again
Update: 2:59UTC - heading into LVEA to reset Noise Eater, second time my shift
Update: 3:14UTC - noise eater reset - X arm IR locked and WFS offloaded
Update: as of 3:33UTC, Initial Alignment is complete
3:44UTC - locking full IFO
6:09 Bubba on his way to the site to investigate instrument air.
16:25 Initial alignment: Input align not showing any IR. Trended IM4 and PR2 after successfull ALS alignment. Still no luck. Going to full lock to further assess the need(s) for continuing IA.
16:35 Bubba on site
16:40 Johnathan called to inform me of a tour that he's meeting on site at 10:00PST
16:45 Both TCS Lasers tripped off.
16:49 Set H1 Observatory Mode to Unknown due to investigation of TCS issue.
16:57 John W on site going to assist Bubba
18:30 Checked on and cycled the TCS lasers
18:38 contacted Aiden and then Alistair
19:00 Granted remote access to Alastair
19:26 Carlos granted remote access
19:45 I spoke to Fil on the phone. It has been determined that there's a hardware error. He's heading to the site to troubleshoot.
23:43 Having an unusually hard time re-aligning arms
16:25 Handing off to Cheryl
1215 hrs. local -> Kyle on site to check on 80K pumps Local temperature are dipping into the "single digits" which means now is an opportune time to investigate if our old problem of over-filling or under-filling of 80K pumps will result from frozen LLCV valve stems is going to be a problem. Recall that in the past this had occurred but could not be confidently attributed to water freezing in the old pneumatic actuators or from water freezing in the valve stem from having got past too loose packing nuts. Or, both for that matter. Chandra R. and John W. had replaced the pneumatic actuators with electric (water tight) ones last spring so now is the first real world test. Initially, I'm on site to verify that the recently elevated exhaust pressures on CP3 and CP4 weren't the result of continuous overfilling -> Inspection of the Y-mid's CP3 and CP4 showed neither was overfilling and both are fine for now (remnant standing LN2 in lines from yesterday's overfill?). By dumb luck, I found that CP6 was indicating a frozen valve stem while I was on site - "Go figure!" -> As found, the local actuator pointer was showing ~30% open and the actuator motor was making noise. The PID output was "railed" at its programmed minimum of 20% open. I had John W. (he and Bubba also on site for separate instrument air issue) switch CP6's level control to manual 50% open -> the motor then went quiet and it stroked open to 50% as per the pointer. Next, I had him change the value to 0% open and the stem stroked closed smoothly until stopping at about the 30% open value (as indicated by the local pointer). I then tested the valve stem packing nut and found that it was completely loose, i.e. no thread engagement. So, even though the local heat tape is wrapped around most of the exposed valve stem, was energized and working, water which made it past the ineffective packing washer was able to freeze nonetheless. This is because this area is about 2" away from the closest heat tape wrap -> I heated up the frozen area with a heat gun and lightly engaged the packing nut. I then had John W. alternate between 100% open and 0% open to demonstrate that the valve could stroke through its whole range smoothly etc.. I setup and am leaving a heat lamp to prevent this unit from refreezing (which it would do, now that water is past the packing washer) and had John W. reduce the Beckhoff PID minimum to 15% open down from its as found value of 20%. 1535 hrs. local -> Kyle leaving site
Man this is hard to read - a consequence of cutting and pasting edits. I think this could be a new record for "run on sentences!"
Ed, Dave B., Fil
The PSL FE Computer tripped off again but this time it was due to the repair efforts on the Rotation Stage Beckhoff chassis. Dave is preparing it for a re-start.
h1psl0 is out of the Dolphin fabric and is powered down, Ed is doing the power cycle of the IO Chassis and powering h1psl0 back up.
h1psl0 models running again, IOP has negative IRIG-B excursion, waiting for it to return to nominal. In the mean time Peter and Ed will recover the PSL via MEDM.
h1ioppsl0 has returned from its IRIG-B meandering. I've cleared all IPC and CRC errors. The board is green again.
