corner station vented

SLOOW vent accomplished. HAM 1 took about 8 hrs, HAM 6 about 7 hrs, and vertex about 9.5 hrs.

All piranis need to be calibrated at atmospheric pressure; they were all over the place. PT180 flat lined at 750 Torr. PT120 drifted to 1150 Torr (known issue and will be replaced). The third gauge on vertex, PT170, plateaued at 607 Torr. PT100 on HAM1 plateaued at 753 Torr, and PT110 on HAM6 at 620 Torr. Note that PT110, 170, 180 are the new style full range gauges:  pirani+hot cathode all-in-one.

Closed all three purge air valves to chambers/volumes.

GV5 AIP high pressure

GV5's annulus ion pump went out of range during the vent today. Known inner o-ring leak somewhere not on beam tube side. Turned AIP controller OFF and also valved out the pump.

H1EDCU_DUST.ini modified to add new dust monitors, will go in on next DAQ restart

Jeff B, Dave B,

I have modified H1EDCU_DUST.ini to add the four new LVEA dust monitors (numbers 3,4,5,30). This will go in on the next DAQ restart.

Shift Summary -Day

TITLE: 09/18 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Planned Engineering
INCOMING OPERATOR: N/A
SHIFT SUMMARY:

Venting of HAMs 1 and 6 is underway today
LOG:

15:35 Richard going to inspect HVAC in LVEA

16:14 Jason going into PSL enclosure

16:17 Gerardo out to HAM1&6 to climb up and close ion pump gate valves in preperation for venting.

16:20 Richard and Peter out near PSL investigating remote DBB issue

16:34 Richard and Peter back

17:06 Jason heading back out to PSL enclosure to test AC.

17:14 Richard heading down arms to check fire panels

17:24 Jason called to say that AC in the PSL is still not functioning properly

17:25 Betsy and Travis back

17:26 Hugh out to EY

17:49 Richard back. He had to reset EX fire panel.

18:03 Hugh back

18:07 Jeff B into LVEA to check dust monitors

18:17 Hugh to EX to let HFD in to veriy fire panel.

18:28 Jeff B out

18:31 Richard out to LVEA to reset PSL AC breakers

18:53 Dave B out to EX

20:00 Jason out to PSL enclosure

20:10 Dust monitor 10 300u RED alarm

20:16 Dave B out to both mid stations

20:45 Dave B back

21:10 Travis out to LVEA to drop off a floodlight

22:13 Dave B back. Said the door at MX between the mech room and the VEA was "propped" open?

 

 

 

 

 

 

 

 

 

 

 

 

HEPI Pump Pressure 45 Day Trends

See attached screenshot.  Things I noted were:

• There is a glitch in both the EX and EY pump pressure on August 22 which likely corresponds to a HEPI trip described in alog 38313. 
• There is a large spike starting around August 12 that trends back to ~nominal which is likely due to HEPI being locked (aLog 38618).  The signal flatlines thereafter. Was the CS pump station turned off after all HEPIs were locked??  I failed to locate an aLog if this is the case.
• There is another glitch in both the EX and EY pump pressures Sept. 17, which is likely due to Hugh restarting the HEPI pump (?, no aLog yet although it was discussed in the morning meeting) after the Sept. 16 power outage (aLog 38675). 

PSL Weekly Report - 10 Day Trends FAMIS #6166

PSL is undergoing quite a bit of maintenance. Power down of the system is quite evident indicating the NPRO replacement and the ensuing power fluctuations during alignment. The Chiller manifold was also replaced and ensuing leaks were mitigated over the following few days.

Times switching from TX/RX PCAL channels for calibration

For the purpose of injecting PCAL lines, there are two channels that can be used: H1:CAL-PCALY_RX_PD_OUT_DQ (for reflected light) and H1:CAL-PCALY_TX_PD_OUT_DQ (for transmitted light). At various points during O2, we switched off between these two channels in C00 and C01 data at Livingston to account for things like PCAL clipping in the RX channel. The exact dates of each change are listed below, as documented on the filter configurations wiki page.

Version C00
Used RX from 116440990 (2016 Sep 14, 16:43:07 UTC) -- 1182618646 (2017 Jun 27, 17:10:28 UTC)
Used TX from 1182618646 (2017 Jun 27, 17:10:28 UTC) -- now

Version C01
Used RX from 1163173888 (2016 Nov 14, 15:51:11 UTC) -- 1181941760 (2017 Jun 19, 21:09:02 UTC)
Used TX from 1181941760 (2017 Jun 19, 21:09:02 UTC) -- now

The corresponding record over at Livingston is in aLOG 35948.

PT-114 alarms

Pt-114 is alarming after being isolated from vertex. The rise in pressure from daily tube warming is causing it to cross over the 5e-9 Torr alarm setting.

CDS computing recovery from Sat power glitch continues

EX timing was reporting error with 4th Duotone slave, and h1iopiscex was in error. I found that the timing slave on h1iscex's IO Chassis was still trying to sync up (flashing LED). This system has a separate DC power supply for the timing slave, part of a past noise study. I power cycled both the timing slave and the IO Chassis, and now the code is running correctly and the timing error has been resolved.

Dust Monitor Alarm Levels

   The dust monitor alarm levels throughout the site have been set to their normal levels. In some cases these levels may need adjusting depending upon what is being done in the space they are monitoring. Please get in touch with Jeff Bartlett before adjusting the dust monitor  alarm levels or with any contamination control concerns or questions. 

ISO Dust Monitor Levels

ISO-Clean	0.3µ Minor	0.3µ Major	0.5µ Minor	0.5µ Major
100	200	300	70	100
1000	2000	3000	700	1000
10000	20000	30000	7000	10000
100000	200000	300000	70000	100000

 

Three large IP valve leaks

Looks like three IP isolation valves are leaking: IP1,2,4. Turning those IPs OFF. Valves on HAM 1,6 look good and sealed.

corner vent started

Kyle, Gerardo, Chandra

Started {slow} venting corner station at 10:16 am local, including HAM 1,6. DP measured -45degC or less at HAM 6 & vertex.

Note:  we don't trust pirani gauge on PT-120 at high pressures based on data from May vent. We are monitoring piranis on PT-170 & PT-180.  

New Dust Monitors in LVEA

 Dave B., Jeff B.,

   Today we enabled the new dust monitors in the LVEA. The new dust monitors add trending data for the chambers being opened. The new monitor trend data will go live with a DAQ restart after HAM1 & HAM6 are vented.

LVEA Dust Monitor Locations   

Dust Mon #	LVEA Location
2	HAM2 Cleanroom
3	HAM3 Cleanroom
4	HAM4 Cleanroom
5	HAM5 Clenaroom
6	HAM6 Cleanroom
10	Biergarten - BSC1
30	Biregarten - BSC3

 

 

 

 

 

 

 

 

 

LVEA Pre Vent Dust Trends

   Trended the dust the dust monitors in the LVEA for the past 30 days, 7 days, and 1 day as a base line for the vent. The three dust monitors in the LVEA are:

   #2 in HAM1/2 cleanroom
   #6 in HAM5/6 cleanroom
   #30 in Biergarten for BSC3

   I have posted the 30 and 1 day plots. The data older than 24 ago, for the 0.3u on DM #2 is bogus. Since there is nothing interesting in the 7 day plot it was not posted.    

Diagnostic breadboard connections

The problem found with the diagnostic boardboard was that the bottom DB25 connector bundle
was loose.  This is because of the hardware involved, which has no jack screws to allow an
adapter to screw into the mating connector and the cable into the connector.

    The connector/cable combo was re-seated.  Changing from remote to local was indicated
by the MEDM screen, whereas previously it remained on "LOCAL".

    This closes work permit 7159.




  Richard / Peter

CDS computing recovery after this morning's power outage

To verify that front ends could be started, I first started all the non-dolphin FECs (except for the mid station PEMs), there were no problems. Before starting h1psl0 I consulted with Peter King to see if there would be any PSL issues when I do so, he said there would be none.

I noticed a timing error in the corner station and found it to be the IO Chassis for h1seib3 (ITMX) which was powered down. Its front panel switch was in the OFF position. Richard and I think this was accidentally switched much earlier, and only caused a problem after the computer was power cycled. Before powering this IO Chassis up, I first powered down the two AI chassis because of the issue with the 16bit DACs outputting voltage when the IO Chassis is ON and the FEC is OFF (after h1seib3 was later operational I powered the AI chassis back on).

I restarted all the dolphin FECs, using IPMI for the end stations and front panel switches for the MSR. Many FECs started with IRIGB errors, both in the positive and negative directions. We know from experience some of these take many minutes to clear, so I continued with the non-FEC restarts.

I consulted with Jim Warner on starting the end station BRS computers (it was OK to do so) and the HEPI Pump Controllers (we will leave them till Monday).

I went to EY, the first of many trips as it turned out. I powered up h1ecaty1 and h1hwsey. I noticed that h1brsey was already powered up, but its code is not running?

Back in the control room, I noticed all EY FECs had the same large positive IRIG-B error, indicating a problem with the IRIG-B Fanout. Back at EY I confirmed the IRIG-B fanout was reporting the date as mid June 1999. After some issues, I power cycled the IRIG-B chassis and rebooted the FECs. The front ends were now running correctly.

At this point I ran out of time. There is timing issue with h1iscex, and the Beckhoff timing fanout is reporting an error with the fourth Duotone slave which I suspect is h1iscex.

To be done:

Start mid station PEM FECs.

power up h1ecatx1 and h1hwsex

Start Beckhoff slow controls code on h1ecat[x,y]1. Start HWS code on h1hwse[x,y]

Investigate Duotone Timing error at EX, get h1iscex running.

Start hepi pump controllers.

Start digital video servers.

Start BRS code.

Start PSL diode room Beckhoff computer.

Was able to use remote desktop to connect to h1ecaty1. The terminal for the EPICS IOC was open, but it appeared that something had not started properly. I used the icon on the desktop to restart the computer. This appears to have worked.

I can not reach h1ecatx1. Will likely need to turn it on locally.

Jeff B. powered on h1ecatx1. I logged in with remote desktop and found the same issue as I had with h1ecaty1 (screenshot attached). I used the icon on the desktop to restart the computer. This appears to have worked.

Burtrestored the FMCS IOC to restore alarm levels:

patrick.thomas@zotws12:/ligo/cds/lho/h0/burt/2017/09/15/06:00$ burtwb -f h0fmcs.snap

All the weather stations except the CS had lost their connections. I restarted and burtrestored the IOCs on h0epics.