1500 hrs -> Hard-closed GV10 1700 hrs -> Adjusted instrument air at GV6's piston from <5 psi indicated to 5 psi -> GV6 is likely not in full contact with its gate O-rings -> may adjust air for full contact tomorrow
The hardware watchdog unit has been installed in the DAQ test stand for testing. All electronics associated with this unit are powered up, wiring has been restored to it's previous configuration for testing.
Right now the SUS and HEPI are untripped but if they don't survive the exit of the Chamber by the ISC crew, could someone untrip the SUS and HEPI? Thanks much--H
160 channels added and 4 removed for a current total of 122,626.
8:49 am, Aaron to Y-End, stage relay for noise eater.
9:04 am, Apollo to LVEA, move in scissor lift into LVEA via the high bay area, to remove pre-filters from HAM5 cleanroom.
9:15 am, Corey, Jax and Keita to Y-End, TMS work.==>out by 11:40 am.
9:52 am, Aaron to LVEA, move rack by HAM4/BSC3 area for TCS.
10:19 am, Jeff and Adres to LVEA, do chamber side work inside cleanroom by HAM4/5.
10:29 am, Filiberto to LVEA, to visit both spool pieces to terminate OL cables.==>done by 11:30 am.
10:35 am, John to X-end, inspection of VEA area.==>done by 10:35 am.
11:01 am, Karen to Y-End, clean.==> done by 11:31 am.
11:02 am, Betsy to LVEA, west bay area, ITM assembly.
11:25 am, Craig C to LVEA, work inside H2 laser enclosure.
12:55 pm, Corey to Y-End, TMS work.
1:00 pm, Sheila to X-End, ==>done by 1:21 pm.
2:16 pm, Apollo, Jeff and Kate to LVEA, door removal from HAM5.==>Jeff and kate out by 3:29 pm, Apollo out by 3:36 pm.
2:40 pm, Kyle to Y-Mid, hard close GV10.
3:34 pm, Cyrus and Jim to LVEA, West bay area work.==>done by 3:45 pm.
The I/O chassis and front-end computer for the h1susquadtst test stand has been powered on. The h1iopsusquadtst model was modified to remove a Dolphin IPC channel to allow the model to run on the computer without a dolphin connection. The computer is currently running h1iopsusquadtst, which should be adequate for testing. The rtsystab file was also modified to include the h1susquadtst computer.
Ham 5 doors were removed and laid on pallets near ham 4/bsc 3 to be transfered over beam tube tomorrow morning.
[Sheila Arnaud]
HAM2 HAM3 ITMX ETMX HEPI L4C watchdog thresholds have been increased to 99000, to avoid other HEPI trips for today (we had one on HAM2 today, and following the ones from friday). It's unclear what kind of ground motion event would cause this kind of trip, but that would be something to investigate.
This threshold increase shouldn't affect the security of the ISIs since the CPS and actuator watchdogs are still set to their nominal safe values.
We seem to have lost H1:ALS-X_LASER_IR_DC during the WFS rework.
Jeff asked for an entry showing the order "make install-modelname" commands were issued yesterday afternoon to see if it has any link to the relocking time of the IMC. Note there is a bug in the file name were the month is actually the hour, this has been fixed.
Mar 18 16:39 install-h1susmc1_2014_39_18_16:39:15
Mar 18 16:40 install-h1susmc1_2014_40_18_16:40:25
Mar 18 17:41 install-h1iopsusb123_2014_41_18_17:41:18
Mar 18 17:41 install-h1iopsusb123_2014_41_18_17:41:34
Mar 18 17:42 install-h1susitmy_2014_42_18_17:42:10
Mar 18 17:42 install-h1susbs_2014_42_18_17:42:45
Mar 18 17:43 install-h1susitmx_2014_43_18_17:43:09
Mar 18 17:45 install-h1susmc3_2014_44_18_17:44:40
Mar 18 17:45 install-h1susprm_2014_45_18_17:45:21
Mar 18 17:45 install-h1suspr3_2014_45_18_17:45:41
Mar 18 17:46 install-h1iopsush2b_2014_46_18_17:46:14
Mar 18 17:46 install-h1susim_2014_46_18_17:46:39
Mar 18 17:47 install-h1susmc2_2014_47_18_17:47:18
Mar 18 17:48 install-h1suspr2_2014_47_18_17:47:52
Mar 18 17:48 install-h1sussr2_2014_48_18_17:48:24
Mar 18 17:49 install-h1sussr3_2014_48_18_17:48:48
Mar 18 17:49 install-h1sussrm_2014_49_18_17:49:10
Mar 18 17:49 install-h1susomc_2014_49_18_17:49:30
Mar 18 17:51 install-h1iopsusauxb123_2014_51_18_17:51:13
Mar 18 17:51 install-h1susauxb123_2014_51_18_17:51:37
Mar 18 17:54 install-h1susauxh2_2014_54_18_17:54:05
Mar 18 17:54 install-h1iopsusauxh34_2014_54_18_17:54:54
Mar 18 17:55 install-h1iopsusauxh56_2014_55_18_17:55:40
Mar 18 17:56 install-h1susauxh56_2014_56_18_17:56:07
Mar 18 17:56 install-h1iopseib1_2014_56_18_17:56:43
Mar 18 17:57 install-h1iopseib2_2014_57_18_17:57:06
Mar 18 17:57 install-h1iopseib3_2014_57_18_17:57:33
Mar 18 17:58 install-h1hpiitmx_2014_58_18_17:58:17
Mar 18 17:59 install-h1iopseih16_2014_58_18_17:58:59
Mar 18 18:00 install-h1hpiham6_2014_00_18_18:00:18
Mar 18 18:00 install-h1isiham6_2014_00_18_18:00:36
Mar 18 18:01 install-h1hpiham3_2014_01_18_18:01:21
Mar 18 18:02 install-h1isiham2_2014_01_18_18:01:47
Mar 18 18:02 install-h1isiham3_2014_02_18_18:02:15
Mar 18 18:02 install-h1iopseih45_2014_02_18_18:02:51
Mar 18 18:03 install-h1hpiham4_2014_03_18_18:03:14
Mar 18 18:03 install-h1hpiham5_2014_03_18_18:03:33
Mar 18 18:04 install-h1isiham5_2014_04_18_18:04:13
Mar 18 18:04 install-h1ioppemmy_2014_04_18_18:04:43
Mar 18 18:05 install-h1pemmy_2014_05_18_18:05:52
Mar 18 18:06 install-h1ioppsl0_2014_06_18_18:06:18
Mar 18 18:07 install-h1pslfss_2014_06_18_18:06:53
Mar 18 18:07 install-h1pslpmc_2014_07_18_18:07:15
Mar 18 18:07 install-h1psldbb_2014_07_18_18:07:37
Mar 18 18:08 install-h1iopoaf0_2014_08_18_18:08:12
Mar 18 18:08 install-h1peml0_2014_08_18_18:08:37
Mar 18 18:09 install-h1tcscs_2014_09_18_18:09:02
Mar 18 18:09 install-h1odcmaster_2014_09_18_18:09:26
Mar 18 18:10 install-h1omc_2014_09_18_18:09:57
Mar 18 18:10 install-h1iopasc0_2014_10_18_18:10:25
Mar 18 18:10 install-h1amcimc_2014_10_18_18:10:59
Mar 18 18:11 install-h1sushtts_2014_11_18_18:11:30
Mar 18 18:15 install-h1ioppemmx_2014_15_18_18:15:36
Mar 18 18:16 install-h1pemmx_2014_15_18_18:15:54
Mar 18 18:16 install-h1iopsusey_2014_16_18_18:16:22
Mar 18 18:16 install-h1iopsusex_2014_16_18_18:16:38
Mar 18 18:17 install-h1susex_2014_17_18_18:17:06
Mar 18 18:17 install-h1susetmx_2014_17_18_18:17:23
Mar 18 18:17 install-h1susetmx_2014_17_18_18:17:32
Mar 18 18:19 install-h1susetmx_2014_19_18_18:19:17
Mar 18 18:21 install-h1iopseiey_2014_21_18_18:21:27
Mar 18 18:22 install-h1hpietmy_2014_22_18_18:22:18
Mar 18 18:23 install-h1isietmy_2014_22_18_18:22:44
Mar 18 18:23 install-h1isietmx_2014_23_18_18:23:13
Mar 18 18:23 install-h1iopiscey_2014_23_18_18:23:46
Mar 18 18:24 install-h1pemey_2014_24_18_18:24:14
Mar 18 18:24 install-h1iscex_2014_24_18_18:24:52
Mar 18 18:25 install-h1odcy_2014_25_18_18:25:11
Mar 18 18:25 install-h1odcx_2014_25_18_18:25:26
Mar 18 18:25 install-h1iopsusauxey_2014_25_18_18:25:55
Mar 18 18:26 install-h1iopsusauxex_2014_26_18_18:26:05
Mar 18 18:27 install-h1susauxex_2014_26_18_18:26:59
Mar 18 18:28 install-h1susetmx_2014_28_18_18:28:17
I had a look at the IMC ODC summary around the times stated above and the ODC reports the IMC is green for the following times (in PDT): 16:30:39 - 16:31:40 17:50:31 - 17:50:32 17:50:36 - 17:50:38 17:51:00+ I've also attached a plot showing the IMC ODC over a 3 hour time period which includes the times of interest (plot starts at 16:30 PDT)
Attached are plots of dust counts for locations 1 and 2 in the end Y VEA. These are minutes trends going back 30 days from 08:00 am PDT March 19, 2014 (February 17 - March 19). H0:PEM-EY_DUST_VEA1_500NM_PCF: Dust counts greater than .5 microns at location 1 normalized to particles per cubic foot. H0:PEM-EY_DUST_VEA1_300NM_PCF: Dust counts greater than .3 microns at location 1 normalized to particles per cubic foot. H0:PEM-EY_DUST_VEA2_500NM_PCF: Dust counts greater than .5 microns at location 2 normalized to particles per cubic foot. H0:PEM-EY_DUST_VEA2_300NM_PCF: Dust counts greater than .3 microns at location 2 normalized to particles per cubic foot. March 5, 2014 Betsy finds contamination on the cartridge, notes that it must have occurred after installation, comments that dust counts recorded at BSC 10 prior to this date are not useful due to the location of the dust monitor there. alog February 28, 2014 - March 3, 2014 No dust counts available, Comtrol found mysteriously unplugged alog February 24, 2014 cartridge installation alog Jeff's plots of dust counts during cartridge installation alog February 21, 2014 ISI wiped down alog February 20, 2014 TMS bagged alog SUS bagged alog December 5, 2013 Dome and north door removed from BSC10 alog
(Alexa, Sheila, Kiwamu) -- a post from yesterday that didn't get uploaded due to alog maintenance
I repeated the IMC intensity noise measurement that was done for HIFOY (alog 7364):
Step 1: Measure the power spectrum of MC RIN via H1:PSL-ISS_PDA/B_CALI_DC_OUT with both the ISS off (REF 7) and on (REF 11) (see 20140318_MCRIN_Data png for these measurements).
Step 2: Calibrate via the following:
IMC_trans_freq_noise = f0/L*PWR*(2+1+1)/c/mMC2/(2*pi*f)^2 * RIN where f0=2.818e14 Hz (red frequency) L=16.4736 m (IMC length, one way) PWR=1.36e3 Watt (Power in the IMC, = 8290 mWatt * Finesse/pi, Finesse=516) mMC2=2.9kg (Mass of MC2 mirror, same as for MC1 and MC2) (2+1+1) (effect of MC2 at 0deg, MC1 at 45deg, MC3 at 45deg) f (Audio frequency)
This calibration is set up in matlab file: calib.m (and returns the HZ txt file). We only use PDB_DC_CALIB since this measurement gives the out of loop sensor noise (meanwhile PDA gives the in-loop). See screenshot for result.
Note: In order to ensure a proper calibration of these channels with the increased power into the MC, I adjusted H1:PSL-ISS_PDA/B_CALI_DC_GAIN such that the tsdavg was 1. PSL TEAM: does this change the gain of the loop?? Feel free to return to the nominal values. The ISS and MC still lock under the new configuration.
(all these files can be found in: /ligo/home/alexan.staley/Public/IMC_IntensityNoise/... I was not able to upload them because the files were too large)
The next step will be to insert this result into the model...TBD
I attached the wrong DTT snap shot. I calibrated PDB (not PDA); however, the DTT snapshot I previously posted was of PDA. I have attached a matlab plot of the dtt power spectrum of PDB.
model restarts logged for Tue 18/Mar/2014
2014_03_18 11:27 h1susetmx
2014_03_18 11:53 h1susetmx
2014_03_18 12:03 h1susitmx
2014_03_18 12:32 h1susitmy
2014_03_18 12:51 h1susetmx
2014_03_18 12:52 h1susitmx
2014_03_18 12:55 h1susitmy
2014_03_18 13:01 h1susbs
2014_03_18 13:05 h1suspr3
2014_03_18 13:07 h1sussr3
2014_03_18 13:08 h1susetmy
2014_03_18 13:12 h1dc0
2014_03_18 13:14 h1broadcast0
2014_03_18 13:14 h1fw0
2014_03_18 13:14 h1fw1
2014_03_18 13:14 h1nds0
2014_03_18 13:14 h1nds1
2014_03_18 13:21 h1broadcast0
2014_03_18 13:27 h1broadcast0
2014_03_18 16:22 h1iopsush2a
2014_03_18 16:22 h1susprm
2014_03_18 16:24 h1suspr3
2014_03_18 16:25 h1susmc1
2014_03_18 16:27 h1susmc3
All restarts expected. h1broadcaster multiple restarts due to memory issues.
Large number of restarts due to Tuesday Maintenance Work
One fan has been secured and there is only one fan running now at X end.
I turned off three gowning rooms at ~10:40 am (2 in the VEA, 1 at the entrance)
Ski turned off one of the two HVAC fans at 10:57.
The TMS lab is still operating - I will ask Corey/Keita about this.
For some reason (maybe related to the CDS boogie man???) the supervised guardian nodes (ie. the nodes running under the guardian supervision infrastructure on h1guardian0) are unable to talk to any of the h1ecatx1 TwinCat IOC channels.
Sheila first noticed this when guard:ALS_XARM was not able to connect to the H1:ALS-X_LOCK_ERROR_STATUS channel. The truly weird thing, though, is that all other channel access methods can access the h1ecatx1 channels just fine. We can caget channels from the command line and from ezca in python. We can do this on operator workstations and even from the terminal on h1guardian0. It's only the supervised guardian nodes that can't connect to the channels.
I tried reloading the code, restarting the guardian nodes, nothing helped. The same problem regardless of which node was used. Note:
I'm at a loss.
There's clearly something different about the environment inside the guardian supervision infrastructure that makes this kind of failure even possible, although I honestly have no idea what the issue could be.
I'm going to punt on trying to spend any more time diagnosing the problem, since I'm just going to chalk it up to the other weirdness. Hopefully things will fix themselves tomorrow.
The other thing to note is that I did an svn update on that computer right before it crashed, it might be worth looking at what was included in the update to see if it changed the behavoir of the IoC somehow.
(As also discussed in person.) This may be due to a difference in the environment setup between the Guardian supervisors, and a login shell. The EPICS gateway processes are still in place on the FE subnet, as we have not changed the data concentrator or FE systems to directly broadcast to other subnets. So, the channel behavior will be dependent on the setting of the EPICS_CA_ADDR_LIST environment variable, specifically whether CA will traverse the gateway or route through the core switch. The problem described sounds a lot like the issue the gateway has with reconnecting to Beckhoff IOCs, if the Guardian processes are connecting to the gateway then this would explain the behavior described. Jaime was going to look at the Guardian environment setup as time permits, to see how it differs from the current cdscfg setup.
some of the digital video cameras used to have names, that appeared in on the links in the H1VIC_DIGITAL_OVERVIEW.adl screen.
The names disappeared sometime this afternoon, maybe related to the deleted files?
The overview is one of the files that went missing. I had an older copy saved elsewhere which I restored so as to provide access to the cameras, but this does not have the (later) name updates. Once the backup copies are retrieved from tape, I should be able to restore the latest version. It was also an oversight that these were not in SVN, which will also be corrected.
I have restored the latest version of the camera screens, they have also been added to the useraps repo in cds/h1/medm, and symlinked in the /opt/rtcds/lho/h1/medm/cds area.
TMS Work (Corey, Jax, Keita, Margot)
Apollo roughly positioned the ISCT-EY Table (via floor markings).
Chamber Floor was first cleaned upon entry for work (did not do an exit floor wipe). Margot then entered the chamber to remove First Contact from inside surface of pair of TMS Viewports (then went out and removed First Contact off outer surfaces). She mentioned "finger prints" on the outside surface of one of the TMS viewports; Margot will document in the DCC.
By eye, we checked position of the table, we ended up pushing the table about 3" west. We then attached ducting between the chamber and the table. Then we went in to check our line of sight from the TMS Table to the Table Periscope. This required us to move the GREEN periscope an inch east, and also the top periscope mirror down 4-6". At this point the laser wasn't quite making it down the ductThis is about where we ended things last evening.
Dust Monitor Check Of Purge Air For BSC10 (Corey, John, Keita)
John whether there was dust coming from the purge air. So I, with handheld Dust Monitor in hand, crawled under the ACB, and took a few measurements from the Dust Monitor (which was running continuously). I took measurements at four spots above the input of the purge air. At the highest point, I had readings which would hover between 1500-2500 counts of 0.5um particles. At the lowest point (with sensor of Dust Monitor right at the input), the 0.5um counts could be kept at basically 0-counts (with flashes of a few hundred possibly. So, it would seem the purge air is relatively clean.
These 2 statements seem contradictory:
"At the highest point, I had readings which would hover between 1500-2500 counts of 0.5um particles. [At the lowest point ...] So, it would seem the purge air is relatively clean."
Maybe can John elaborate at where he thinks the high counts are "coming from" if not the purge? Is it just that the air is turbulent in chamber and stirring up the 1500-2500 counts already in the chamber?
Corey saw high background levels in the beam manifold but was able to drive the particle counts to zero by moving the detector close to the purge port at the floor of the beam manifold. I walked around the VEA sampling and found low levels throughout. Counts inside cleanrooms were close to or equal to zero except at the open BSC door. There was activity here as well as equipment staged. Corey was inside and Keita was outside at the BSC entrance. The overhead work platform reduces the effectiveness of the clean room in this location. Inside, the arm cavity baffle obstructs access to the beam manifold so any work in the beam manifold requires a person to laydown and slide under the baffle. This may very well abrade clothing. I recommend we establish a horzontal clean flow as we have while working in HAM chambers.
A reminder that the purge air can only provide 25--50 cfm of air flow into the chamber. In a 6 foot diameter tube (beam manifold) this translates to air velocities of only 0.6 to 1.2 feet per MINUTE. Think how far you walk in a minute.
My impression is we have a reservoir of particulate in the vacuum chamber from the series of operations which have taken place - for example there have been two cartridge installs and one removal. Also this cartridge is an early assembly - probably assembled prior to some of the "in process" cleaning steps we have adopted.
Yes, I should elaborate a little (was quickly entering alog during Morning Meeting). So we did measure counts while I was in BSC10. And seemed like we had steading counts in the several thousands [for 0.5um counts with continuous sampling]. When I took measurements along the Purge Air plume, I would get up to 2500 at the most at the top of the plume (6' high). As I went closer and closer down to the Purge air inlet, the counts started to drop. And it was zero right at the inlet.
So the picture looked as though we have a baseline of particles floating in the chamber. And in the turbulent air above the Purge Air inlet the counts waiver a bit, but counts decrease the closer you are to the inlet. So above the purge we have particles moving around more (vs further away from purge these particles are more "statically" floating...perhaps they are more on the floor when someone like me isn't shuffling them into the air).
Basically we have particles all over the surfaces in the chamber/tube. They may be gently floating around or resting on the floor. They get rustled around when we work in-chamber & also get blown around and away from the Purge Air inlet. We need to remove these particles...which I know is obvious and daunting.
We're going to coninue wiping these particles on the floor toward door, but not sure what that does. Hopefully, particles get attached to our wet wipes, but I wonder if particles just get pushed to the edge of the floor and then fall over the edge of the temporary floor and then rain down to the bottom of the chamber. Sad sad.
Are these raw or normalized counts?
