I noticed that the dust monitor 10 (beer garden) was stopped, trending showed it was stopped about 09:58 PST Tuesday Dec. 20. After talking with Jeff B, restarted it. It's now faithfully reporting dust in the LVEA again.

• 08:38UTC restarted lock clock. It had been frozen since the last 4+hr lock
• EX Temp is High Alarm @ 14:55:00, 15:30:15. Some temp data is listed below in screenshots.
• 15:58 Out of Observing to run a2l script while Livingston is doing some WFS investigations regarding their short lock stretches. Some measurement progressions are listed below including the latest post-a2l script screenshot.

All trends appear normal. Diode powers tracking humidity trends as usual. Everything else is relatively nominal.

TITLE: 12/21 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC

STATE of H1: Observing at 74.4192Mpc

INCOMING OPERATOR: Ed

SHIFT SUMMARY: Been Observing for 4 hours and counting. Quiet shift so far. Lockloss a little bit after Robert went in to LVEA (I forgot to log this one), coincide with a 6.7M earthquake in East Timor but the earthquake didn't show up that high on the FOM (only 1um/s max). 

LOG:

0:27 Kyle back from mid Y

0:30 Kyle going back out

0:44 Robert out doing opportunistic injection in PSL and LVEA (LLO is down).

1:55 Kyle back

02:54 Observe

3:11 Out of Observe. LLO is down so Jeff is doing an opportunistic calibration work.

03:14 Lockloss. ASC signals went unstable. Jeff suspect high useism.

04:12 Observe

Carlos, Jim B, Dave, Nutsinee

Follow up of WP6382

I took advantage of the earlier down time to test the HWS stream images script and swapping CLink around. The original set up after Carlos installed the second PCI-e card was HWSY CLink was plugged into the new card and HWSX CLink remained the same (plugged to old card). HWSX streamed live images without problem while HWSY can't get any sensible image at all (see attachment). So I tried a few more configurations:

• HWSX CLink plugged to the new card, HWSY plugged to the old card
  • Result: HWSY worked, HWSX didn't
• HWSX and HWSY CLink both plugged into the new card
  • Result: None worked
• HWSX and HWSY CLink both plugged into the old card
  • Result: Both worked (just like how they were prior to adding the second card)

We have come to the conclusion that the spare card might be busted. The next thing to try is to order a new card and try again. This way we will know for sure that the card actually works. We don't know where the spare card came from and whether or not it has been tested.

This has not been attempted previously to CP3 or CP4 though it is not a new or original idea.  

Both CP3 and CP4 are thought to be experiencing the same issue, i.e. an obstructed level sensing line.  This could be due to the accumulation and clumping of "snow flakes" of water ice.  It may be that the relative buoyancy in LN2 of these "snow flakes" is concentrating them near the bottom of the LN2 pump which is also where the lower sensing line enters the pump and samples the LN2.  With an increasingly dense concentration over time it is easy to imagine these "snowflakes" interlocking and forming clumps which eventually restrict the ID of the sensing line (the "cholesterol" theory!) to the point of complete obstruction.    

In the past we had applied UHP N2 (various pressures/durations -> Max. of 180 psi) to these clogged lines in an attempt to "push" this obstruction back into the 80K pump.  These attempts gave no indication of a resulting movement of the obstruction nor flow of the applied UHP N2.  Today, while changing the most recently depleted cylinder of UHP LN2 (leak into the room), I decided to reconnect the sensing line to the local differential pressure gauge.  I noticed that it indicated a differential pressure of 5" of water.  I repeated the process of decoupling the line from the gauge and exposing it to atmosphere for a period then reconnecting it to the gauge.  Each time the pressure difference increased by ~0.5" of water.  This got me thinking..... 

Considering the relatively large mass of water that can be extracted from food in the freeze drying process while exposed to vacuum and in such a relatively short amount of time, I discussed with John W. the risks and possible benefits of applying vacuum to our clogged CP4 line.  The hope is to encourage the sublimation of the "ice plug".  It was decided that applying a vacuum to the sensing line would be low risk at this point.   

So, late in the day, I hooked up a small diaphragm pump and pumped on the sensing line for a few minutes.  After reconnecting it to the differential pressure gauge and CDS level transducer I found that the indicated pressure difference had increased.  Repeated cycles produced repeated benefit (see attached image).  Of particular note is that an audible release of built-up gas could be heard each time the sensing line connection was loosened to do the next round of vacuum pumping.  This suggests that the CDS indicated pressure difference is due to a real pressure difference and that, perhaps, the vacuum pumping has established a flow path through the plug!  Or, that the ice plug has been pulled into a warm area.

Gerardo and I will continue tomorrow.  Chandra R. will be back after the new year. 

Resumed pumping 2K IMC annulus ion pump volume with pump carts during the maintenance period today.  Energized ion pump controller after an hour or so of pump cart-only pumping and the combination got the ion pump current down to 5ma -> Left ion pump controller on but had to shut down the assisting pump carts at the end of the maintenance period.  As of this writing, it looks like the ion pump current is just now coming back on scale so, perhaps, additional assisted pumping won't be required.

WP6399 cdsfs0 raid firmware update

Carlos:

cdsfs0 raid controller card firmware was updated. The /ligo file system was offline for a short period of time

WP6382 add second camera card in h1hwsmsr

Carlos, Nutsinee:

The spare HWS camera pci-e card was installed in h1hwsmsr and one of the camera was moved over to it (see photo)

WP6407 start robo-alog code

Jonathan, Ryan, Dave:

The new robo-alog entry system was tested and is ready for use by the vacuum autofill process.

Attached:

• HEPI pumps - 45 days
• Temperatures - 45 days
• HEPI pumps - 10 days
• Temperatures - 10 days

Some drop in presure - possibly due to temperature.

Now the test timing system in the EE lab is powered down, h1ecatc1plc1 reported a difference for the master on port 7. Since this is a test system, we unmonitored this channel.

This is a test of posting from script0 using robot credentials.
Have a very nice day.

The IM alignments were adjusted many times in the commissioning ramp up to O2, so IMs have not had nominal values for some time, and DIAG_MAIN values were not consistant with the current IM values.

I adjusted IM1-3 alignments after taking some measurements during maintenance today, and in looking at the IM alignment history.

I saw the IM1 pitch had bee restored to the DIAG_MAIN value yesterday, and this was not a correct value.

The current nominal values for IM1-3 OSEM alignments, as deemed good on Dec 8th at 09:00UTC, after a discussion with Jenne, are:

These are also listed in the IM Nominal Alignment medm (IOO sitemap list), and are updated in the DIAG_MAIN code.

To check IM alignment:

• use the procedure in the "Recovering IM Alignment After ISI Trip" wiki page

Today, I performed the LVEA sweep after everyone finished their Tues maintenance work.  I confirmed everything on the list, except for 2 of the 3 crane parking placements - we didn't use any cranes today and they have been parked in the same place for over a week.  I also only glanced at the electronics racks, since we also had done a nice sweep to disconnect miscellaneous stuff from racks prior to the run starting.  I did not see any new dangling pieces of test equipment or other interesting looking stuff.

I did unplug the West bay phone along the wall - likely on since the start of O1 since it's a somewhat unfamiliar phone to people.  Here is the full list of LVEA phones I found and their status as of now (which should be added to the sweep checklist):

1) Inside H1 PSL enclosure - OFF

2) On tool box or table near H1 PSL enclosure - OFF

3) West bay along wall near crane parking sign - now OFF

4) On table near HAM7 East door - OFF

5) Along wall near HAM6 North - OFF

6) In Squeezer Bay on table - OFF

7) Back corner of electronics room, on floor - OFF

8) I suspect possibly one is inside H2 PSL enclosure - called it, didn't see it on HAM7 mic, unconclusive if it exists...

The original ETMY PUM coil driver (S1102652) was reinstalled this morning. Two hour lock shows little or no change in the glitch rate. The AI chassis controlling the PUM chassis was also power cycled.