Reports until 10:52, Sunday 02 October 2016
H1 PSL (OpsInfo, PSL)
corey.gray@LIGO.ORG - posted 10:52, Sunday 02 October 2016 - last comment - 10:50, Friday 07 October 2016(30150)
PSL Recovery From Flow Sensor Trip (and Being down/cold for 10+hrs)

(Corey, Jason on phone)

Last night, Nutsinee came in early for her OWL shift & just before midnight the PSL tripped.  We opted to leave it tripped for the night to wait for Jason to walk us through recovery procedure.  Atleast for now, we always want to run through this procedure with one of our PSL people (Jason or Peter K).  Here are my rough notes:

Now on to "Aligning" on Observatory Mode!

PSL Status NOTE:

Comments related to this report
dennis.coyne@LIGO.ORG - 12:21, Sunday 02 October 2016 (30153)

The Xtal chiller has tripped 3 times in the last week:

  • 9/25 (29964) - added 200 mL water - "normal due to known slow leak"

then 4 days later:

  • 9/29 (30063) - added a "few 100 mL" water - water sprayed onto door

now 3 days later:

  • 10/2 (30150) - "cap blew off" - added 375 mL

The last two events imply an overpressure condition. The problem seems more significant than replacing lost fluid from a slow leak.

Was the cap which "blew off" in today's event the "bleeding cap" (section 4.2.2), the "filter sleeve cap" (section 6.2) or the "filler pipe cap" (section 6.3 of T1100374-v1, "200 W Laser Crystal Chiller Manual")?

 

BTW, LRA = Long Range Actuator; see section 7.2 of  T0900641-v5, "Under Manual 200 W Laser".

matthew.heintze@LIGO.ORG - 12:51, Sunday 02 October 2016 (30155)PSL

From my quick look through the alog I think the PSL has tripped more than this (for LHO to concur). From what I can see it has gone off 11 times in the last week (thats as far as I looked back).

 

I'm not sure if all the same problems, but alogs reporting the PSL laser off are LHO alogs:

10/2/2016

LHO alog 30160

LHO alog 30154

10/1/2016

LHO alog 30146

LHO alog 30143

9/30/2016

LHO alog 30118

LHO alog 30086  (due to power glitch)

9/29/2016

LHO alog 30076

LHO alog 30063 (this alog reports two different  instances of the laser going off)

9/27/2016

LHO alog 30016

9/25/2016

LHO alog 29964

 

The filler pipe cap popping out is a known thing (happens all the time at LLO) when the chiller turns OFF. At LLO at least this has not shown to be due to any problem (just a consequence for whatever reason when the chiller is turned off).  Its why we try to not turn these chillers OFF if can help it as they "burp" water over the floor and pop these fill caps even when trying to restart

 

I have as one of the main agenda items of this Wednesdays PSL meeting to discuss this problem and see if we can work out whats going on. Some statistics on how many times happened in say the last month or two, what the PSL trip was attributed to, and how many times happened before and after the chiller swap (to see if accelerating or at the same rate), would help this. Im not sure if FRS fault reports have been made for each laser trip to make this search easy for us remote to the site to do and work out how much lost observatory time we have had due to this issue.

corey.gray@LIGO.ORG - 12:58, Sunday 02 October 2016 (30157)PSL

Yes, Dennis, as Matt says it's the Filler Pipe Cap that pops off.  We have had (3) trips since Friday evening (so Matt is probably right about there being more trips over the whole week).  OH, and I should correct ourselves here because we just had a 4th Weekend PSL trip (this was just after I had H1 at NLN for 15min.  This time the cap was blown off and there was a puddle on the floor.

Able to get back in 30min this time (vs 60min this morning).

OK, back to locking.  

corey.gray@LIGO.ORG - 13:42, Sunday 02 October 2016 (30158)

Another Note:

Something I wanted to add about the chiller was that when filling it, I noticed quite a bit of turbulence in the fill pipe.  And you could see a air bubble vortex/tornado in there.  Something we probably don't want if air bubbles are postulated as a trigger for flow sensor trips.  

matthew.heintze@LIGO.ORG - 13:48, Sunday 02 October 2016 (30159)

We see these bubbles as well in the chiller fill pipe at LLO with no chiller trips due to it (hopefully haven't jinxed myself). Perhaps post a movie of it so can see if looks the same as here

corey.gray@LIGO.ORG - 14:49, Sunday 02 October 2016 (30163)

Jason & crew will be investigaing tomorrow.  We should ask them to record a video of it.

matthew.heintze@LIGO.ORG - 05:21, Monday 03 October 2016 (30165)

Movie of the water turbulence of LLO's crystal chiller fill tube posted at LLO alog 28397

jason.oberling@LIGO.ORG - 10:50, Friday 07 October 2016 (30304)

I took a video, but I can't post it; my phone only takes video in .mp4 format, which is apparently not a valid file type for upload to the alog.  Huh.

I attached a still from the video to give you some idea of what we've been seeing here for the last few weeks.  It's appearing to fluctuate though; the video was taken on Monday, 10/3/2016, but this morning our fill port looks very similar to what's seen in Matt's video of the LLO crystal chiller fill port.

Images attached to this comment