Walked in to see Cheryl & Jason on the phone & our PSL down (since 11:20pm local time).  As I got the hand-off from Cheryl, Jason had me log in a PSL computer (on ops station, opened a terminal and used vncviewer).  On the h1pslctrl0 desktop we could see that the Diode Chiller was OFF.  We toggled it ON, but it would immediately turn off.  One possible reason for this could be water level, so Jason had me go to the PSL Chiller room to check things.

Puddle Of Water Below PSL Chiller Discovered/Inspected (12:27-1:12am local time)

Upon entering the PSL Chiller room, I saw a puddle of water on the floor underneath the PSL chiller rack (see photo 1 & 2); the door ended up sloshing the little puddle when I opened it.  I looked at the Diode chiller (the one on bottom) and on its panel we had a RED Error light & the following notes:  (1) Flow Sensor 1 Error & (2) Low Level warning (see photos 3 & 4).

I tried to inspect (photos 6&7) the Diode Chiller to see where exactly this water was coming from, but it was hard to determine.  The panels on both sides of the chiller are removed, so one can somewhat easily inspect the innards of the chillers.  There did not appear to be any active leaking.  I looked at as many plumbing connections as possible, and did not observe any areas which looked visibly wet and a source for the leak (but there is the possibility this puddle could have been there for while and the culprit could have dried up to a point to hide its guilt!  The Diode resevoir has water in it (see photo #5)...in other words, it's not empty...but it's probably at a "low level" as indicated on the front panel.

At this point, I came back to let Jason know.  He stated this is a serious enough failure that multiple PSL staff would be needed to address the issue.  

Notified Keita (who is also the scheduled LHO Site Advocate) of the situation (also sent text to Jeff J & TJ).  Jason sent an email to JeffB to have him prep a spare chiller for a swap in the morning so they could get started first thing.  Asked Jason if the Diode Chiller was in a safe state as it is for the night and he said it should be.  

I contacted LLO & Virgo operators to asses them of our status.  

FRS Ticket #13939 submitted.

Status Summary:

H1 is DOWN until the PSL Chiller can be brought back online.  The plan is for the PSL team to address this issue first thing in the morning.