John and I visited CP5 before lunch today to investigate why the liquid level is so noisy (compared to CP6). We verified wires were tight at the controls rack, and eventually made our way to the LL transducer. We closed the exhaust at transducer and the flow stabilized suggesting the instability is caused by a real pressure differential and not electric. We did not check the LLCV pneumatic actuator. After trending the numbers this evening, looks like we did a real number on the system. See plots attached. The % valve open is ranging full scale and LL full spans 90-95%.
Could this be related to the midstation air compressor replacement?
Richard just reset the PID values (same values). The LL seems to have stabilized (well, to its prior stability which is still relatively noisy). We will watch it throughout the day.
Per https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=26797 Set valve 25% open in manual mode. It currently reads 94% full. Tomorrow we will transition to PID.
Gerardo, Kyle We torqued BSC4's dome bolts and found many that were very loose. We then valved-out the pump cart and the ion pump responded by temporarily "railing" but has since come on-scale on its own - i.e. we fixed the outer O-ring leak -> The pump cart was then shut down (this pump cart has been running near the SW corner of BSC4 for the past few weeks and shutting it off, finally, is "very exciting" for us!). During this process, it looks as if the signal cable to the Diagonal Volume's pressure gauge-pair, PT140, was disturbed resulting in an anomalous reading from the Pirani gauge. This then tripped off the Cold Cathode gauge -> We were able to "wiggle" the cable etc. and get them to resume normal readings. Additionally, we determined that HAM11's ion pump needs to be replaced but are electing not to replace it as HAM12's ion pump can keep the combined HAM annulus volumes at adequate vacuum for the time being. Currently there are two pump carts running in the LVEA by HAM11 and HAM12, these will be shut down in the next day or two.
Yesterday (Monday 25th April) the MY vacuum controls system was upgraded to Beckhoff. Today (Tuesday 26th April) both LVEA systems (LX and LY) were upgraded to Beckhoff. I have performed the following on all three systems:
* = in the new system CP pump levels cannot exceed 100%, therefore I reduced the HIGH ALARM limit from 100% to 99%
still to do:
I updated the control room alarm handler.
Reset ITMX.
15:02 UTC Peter to H1 PSL enclosure 15:06 UTC Turned off BRS sensor correction at end X and end Y for Karen to enter the VEA to clean 15:19 UTC Gerardo shimming CP1 LLCV and CP2 LLCV in preparation for Beckhoff vacuum controls upgrade 15:23 UTC Filiberto taking tools to LVEA for Beckhoff vacuum controls upgrade 15:27 UTC Joe to LVEA to charge batteries 15:27 UTC Jeff K. starting charge measurements on ETMX and ETMY 15:30 UTC Bubba and Nicole to LVEA 15:33 UTC Sprague through gate 15:33 UTC Richard to LVEA to work with Filiberto 15:33 UTC Betsy and Travis to LVEA and optics lab 15:42 UTC Ed to LVEA to work with Richard and Filiberto 15:59 UTC Joe out of LVEA, taking Sprague to mid and end stations 16:05 UTC Carlos working on DMT network 16:12 UTC LN2 delivery through gate 16:12 UTC Karen out of end X, getting shoe covers from corner station 16:34 UTC Karen done dropping off shoe covers at end X, going to end Y 16:39 UTC Beckhoff vacuum controls upgrade done at LY 16:46 UTC John and Chandra to mid X to look at signals 16:47 UTC Hugh to end stations to check HEPI fluid levels 16:54 UTC Jeff K. done charge measurements 16:56 UTC Joe back from escorting Sprague 17:19 UTC Kyle to end Y to record number for property audit 17:21 UTC Gerardo to BSC4 to tighten bolts around dome 17:22 UTC Paradise water delivery 17:24 UTC Karen and Chris done at end Y 17:30 UTC Ryan done WP 5831 (internet back) 17:37 UTC Kyle back from end Y Richard and Filiberto starting Beckhoff vacuum controls upgrade at LX 17:56 UTC John and Chandra back from mid X 17:59 UTC Keita escorting film crew to X arm mid point beam tube tunnel 18:03 UTC DAQ restart for LX and LY Beckhoff vacuum control channels 18:10 UTC Joe to LVEA 18:15 UTC Jim W. taking ETMY ISI and HEPI down for measurements 18:28 UTC Joe back 18:44 UTC Keita done from escorting film crew 18:45 UTC Gerardo done at BSC4 18:46 UTC Vending machine delivery 19:42 UTC Filiberto to LVEA to work on dust monitor wiring 19:42 UTC Carlos done 20:28 UTC Richard to look at CP1 20:29 UTC Filiberto done with dust monitor work, going to beer garden to see what is needed for cabling PT170 and PT180 BPG402 gauges to Beckhoff vacuum controls 20:35 UTC Bubba replacing mid X instrument air compressor 20:36 UTC Peter done in H1 PSL enclosure 20:45 UTC Travis to LVEA drop off bins and parts 20:59 UTC Travis back 21:23 UTC Filiberto to LVEA to setup for pulling cable 21:45 UTC Filiberto pulling cable from LX vacuum rack to beer garden 21:58 UTC Jeff B. to LVEA to work on dust monitor wiring 23:18 UTC Filiberto done
TITLE: 04/26 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Planned Engineering
OUTGOING OPERATOR: Patrick
CURRENT ENVIRONMENT:
Wind: 16mph Gusts, 8mph 5min avg
Primary useism: 0.05 μm/s
Secondary useism: 0.22 μm/s
QUICK SUMMARY: Maintenance Day. Commissioners have the IFO currently.
Attached are some data traces from today's work on the power stabilisation. After tweaking the alignment of the ISS AOM, I found that the maximum diffracted power in the first order was 4.1 W when the offset slider was at 10%. aomfreqinput.png shows the 80 MHz into the AOM as things were - a 9 dB attenuator was in the path. 0.28 Vrms corresponds to about 1.6 mW. The AOM driver input should be around 6 dBm. aom[5-7].png show the AOM driver input with 5-7 dB attenuators respectively. A 5 dB attenuator was left installed. The 0.45 Vrms corresponds to ~6 dBm. The resulting diffracted light is shown in DiffPwr.jpg. This hopefully will solve the saturation issues observed earlier. rpn2.jpg shows the output of PDA and PDB. PDB being the one used as the sensor for the loop. The agreement between PDA and PDB is okay for the free-running spectra. With the loop closed, the agreement is not so good below 3 kHz. Increasing the servo gain, increased the noise beyond ~10 kHz. AOMControl.png is the spectrum of the drive voltage to the AOM when the servo was locked to yield the above noise measurement. ISSTF2.jpg is the transfer function measurement. UGF is ~58 kHz with a phase margin of ~25 degrees. Obviously plenty of work to do in order track down the source of the excess noise. Hopefully the power stabilisation will not be so fickle.
I changed out instrument air compressor #2 at M X today.
Today we completed the installation of the Beckhoff Vacuum System upgrade. The last two chassis installed were LX and LY. One hiccup was a blown fuse on the Fill control valve on CP-1. This was most likely caused when working on the Watchdog circuit the connects to the 24V supply for this valve. This caused an overfill of the pump. May have to leave in manual fill or control at 99% for the night. Next step will be to add the new Inficon 402 gauges PT170 and PT180 and annulus Ion Pumps .
Jeff B., Patrick T. h0dust is the virtual machine that Cyrus created to run the software for the Lighthouse dust monitors in the H1 PSL enclosure: https://lhocds.ligo-wa.caltech.edu/wiki/h0dust I connected to h0dust over VNC. I closed the two EPICS IOCs. I stopped the dust monitors (which are not actually connected) and disabled the network connections in LMS Express. I closed LMS Express. I shutdown Windows.
Rich and I are trying to figure out what we can use the BRS for more than tilt subtraction of the ground seismometer. Today, we looked at coherence between the BRS and ISI rotational dofs and it doesn't look promising. To check this, we took the chamber to offline, so we could compare the ground rotation seen by the ISI to the motion measured by the BRS. The first plot shows the coherence between the BRS and the ISI, and, sadly, there doesn't seem to be any in RX. Brown and pink are the coherence to the ISI's T240RX and CPSRX, light blue is the T240Y to the BRS RX. The next plot shows what is working well, the tilt subtration from the Y ground STS. Red is the coherence between the BRS and the corrected STS signal, blue is the coherence between the BRS and the uncorrected STS signal, below .1hz the coherence pretty much disappears on the corrected signal until about 10mhz where the subtraction makes the STS signal worse, which we expect. The last attached plot shows the different spectra, calibrated into rad or m, as appropriate. Not a whole else to add, other than the wind was moderate (~10mph) during this measurement, so we probably should expect more coherence with higher winds. Friday looks like a more "promising" forecast, with respect to winds.
J. Kissel, B. Weaver, T. Shaffer I've - measured a new charge data point for 2016-04-26. - processed all measurements that had not been processed over the past few months of measurements - fixed the NDS problems that Betsy and TJ had been having preventing them from plotting the trend of the effective bias voltage As such, it has now become blatantly obvious that between the last time we intentionally flipped the bias voltage on both ESDs in order to mitigate charge accumulation (on Feb 15th; see LHO aLOG 25575) both ETM ESDs have had their settings reverted within a month of the flip, and are now continuing to charge in the wrong direction. Further, it looks like the ETMY ESD bias voltage was reduced by half (albiet still on the wrong direction) somewhat recently on Apr 14th. What's even more worrisome is that neither the ETMY LOCK gain nor DRIVALIGN gain -- which should be used to compensate for the strength change resulting from a reduced bias -- were changed when it was reduced. At least, it appears as though that as we've been commissioning ASC stuff for the past month or so, we haven't been transitioning to ETMY anyways, so it may not have mattered (*phew*) and also why it has gone relatively unnoticed. As such, we should re-flip the bias sign on both ETMs, to what we flipped them to on February and I recommend with flip EY back to the larger value of bias voltage, so that the opposing sign of charge accumulates and the net charge decays back towards zero faster. I've done a good bit of aLOG, snap file time-stamp, and EPICs records sleuthing and have conclude that the flips were unintentional settings loss, as a result of us having SAFE, DOWN and OBSERVE.snap files in concert with several h1susetm[x,y] model restarts that were unplanned or after-thoughts (namely the HWWD install LHO aLOG 25860; and PI model install LHO aLOG 26200). No one is to blame here, it's just a testament to how hard it is to track and control settings during heavy commissioning, especially when we need to make a decision on how many of the three of the SDF files need updating. Kiwamu concisely summarizes the steps to flipping the bias in LHO aLOG 25575, and we'll work with the commissioning team to do so as soon as conceivably possible. However, what Kiwamu failed to mention is that we need to update the H1SUSETMX, H1SUSETMY, and H1CALCS SDF snap files, and do so in the SAFE, DOWN, and OBSERVE snap files. Attached are the charge trends and a zoom of when the EY bias was reduced on Apr 14th.
Back by popular demand (and a working PSL), here are the past 7 day trends. There has been much work going on inside the enclosure so interpretation of these reports will be left entirely to the PSL team leads. Stay tunes for DBB scans for both low and high power coming to a control room near you!
WP 5839 Updated the version of dataviewer to dv-3.0, which addresses CDS bugzilla 832, fixing the previously non-functional "-r restorefile" command line option. Users may now specify a previously saved XML file on the dataviewer command line. As an example the command dataviewer -r Initial_Alignment.xml starts dataviewer, and reads the file Initial_Alignment.xml after connecting with the NDS server. No search path is used for the file, so either the xml file must reside in the current directory from which dataviewer is started, or a relative or full path must be specified for the file.
Jeff K., Patrick T. 15:06 UTC Turned off BRS sensor correction at end X and end Y for cleaning at end stations
THE LVEA IS NOW LASER SAFE
I did a set of tests with the guardian node. The codebase is in a state that should be ready for Jamie and I to set it up tomorrow on the guardian script machine. Going forward things to do are: * Update docstrings * Install glue, gracedb, and grid credentials on guardian machine * Plan out how to run the gracedb process and get robot certificate * Do series of injections with guardian node on guardian machine - test full injection pathway, test killing active injection, test reloading schedule, test multiple injections in a row, etc. Below I outline the tests I did. How to do command line tests with guardian daemon Can now do the following tests on the command line at a LHO workstation: * To test reading schedule and finding the next injection: guardian INJ WAIT_FOR_NEXT_INJECT * To test gracedb event creation: guardian INJ CREATE_GRACEDB_EVENT * To test awg and inject a signal from schedule into the detector: guardian INJ CREATE_AWG_STREAM INJECT_CBC_ACTIVE * To test schedule validation script: PYTHONPATH=/opt/rtcds/userapps/release/cal/common/guardian:${PYTHONPATH}; python guardian_inj_schedule_validation.py --ifo H1 --schedule /opt/rtcds/userapps/release/cal/common/guardian/schedule/schedule_1148558052.txt --min-cadence 300 NOTE: You will need glue and gracedb python packages to run some of these tests, and these packages are not system-installed on workstations in the control room. And for gracedb upload testing you need the grid credential tools which are not on LHO workstations. And for gracedb upload test you need to make sure dev_mode is False. Test injections Injections from last night are in aLog 26749. Today I continued with some more development tests. Injections that are constant amplitude of 1e-26 for 1 second duration into H1:CAL-PINJX_TRANSIENT_EXC; start time of the injections are: * 1145554100 * 1145555100 * 1145555700 * 1145560262 (i) Call to awg works and injection goes into INJ-PINJX_TRANSIENT_EXC. (ii) Injections logged correctly and meta-data is propagating through infrastructure to inform the searches. Can see the hardware injection tests done with the guardian node on the detchar summary pages. The first three not logged with a injection type, eg. BURST, because in initial tests just wanted to correctly use the awg module. Can see thereafter the injections were flagged with a type in ODC and this propagates to the low-latency frames for the online searches and the segment database for the offline searches. Can see attached plots for ODC segments and segment database segments. (iii) Destroying a node with an open stream that has trasmitted data to the front end does not perform the injection. (iv) The gracedb upload functions have already been tested. Today I re-checked the functions and here is an example gracedb event that was uploaded T235981. Adding messages to the event log on gracedb was also tested again, notice the "This is a test." message on the T235981 gracedb page. (v) Schedule validation script updated and tested. Codebase developments Some more changes: * There is now a dev_mode in the code to run the tests mentioned in the section above. At the moment this does two things (i) ignores to check if the detector is locked, (ii) ignores gracedb for now until we get the robot certificate sorted out, and (ii) waits in the INJECT_CBC_ACTIVE state instead of the AWG_STREAM_OPEN_PREINJECT state because we need to avoid jump transitions for the command line test above. * Schedule validation script works again (https://redoubt.ligo-wa.caltech.edu/svn/cds_user_apps/trunk/cal/common/scripts/guardian_inj_schedule_validation.py). One thing of note is that guardian does not allow subprocesses to be created by states so the subprocess managment that I had written will not work with guardian. So right now once the injection starts the code will wait for the injection to finish, this is just the implementation in the awg package (see awg.ArbitraryStream.close); it can only be killed by stopping the node.
I've also renamed the base module (INJ.py) to something less generic, it is now CAL_PINJX.py. See: https://redoubt.ligo-wa.caltech.edu/svn/cds_user_apps/trunk/cal/common/guardian/CAL_PINJX.py So modify the examples in this aLog entry as appropriate, ie. guardian INJ becomes guardian CAL_PINJX.
Chris B., Jamie R. Started up the node with guardctrl start CAL_INJ. Used guardmedm CAL_INJ to control the guardian node. Did a variety of tests with the hardware injection guardian node, these all passed: * Tested killing injection before injection awg call is active by requesting KILL_INJECT. * Tested killing the injection during awg.ArbitraryStream.close call, ie. inject is in active state, by requesting KILL_INJECT. * Tested scheduling injections minimum number of seconds apart to make sure guardian picked the correct injection. * External alert happened while injection was scheduled, aborted injection successfully from AWG_STREAM_OPEN_PREINJECT to ABORT_INJECT_FOR_EXTTRIG. Commented out this check to continue working. * Tested out of order schedule file. * Tested FAILURE_READ_WAVEFORM, eg. waveform file does not exist. * Tested all injection states (INJECT_CBC_ACTIVE, INJECT_BURST_ACTIVE, INJECT_STOCHASTIC_ACTIVE, INJECT_DETCHAR_ACTIVE). * Tested that injection does not go into the detector if we turn off dev_mode so that it checks that detector is locked. * Injection start, injection end times, injection outcome values are all being set on MEDM screen. Made another failure mode. If the call to awg.ArbitraryStream.close is too close in time to the start of the injection, then there is a error. Added FAILURE_DURING_ACTIVE_INJECT. awg returns a generic AwgStreamError so without doing some hacked parsing of the error message, there's not much to differentiate why it failed during the function call. None of the gracedb functionality was tested during this, since we need to create a robot certificate still.
After doing a few more tests, I've started scheduling a long series of injections. The schedule file is here: https://redoubt.ligo-wa.caltech.edu/svn/cds_user_apps/trunk/cal/common/guardian/schedule/schedule_1148558052.txt
Injections were still going this morning from last night as expected, every 400 seconds. Attached is an hour of last nights injections. Also I've attached a zoomed in plot on the fast channel for one injection to check the timing of the start of the injection. Looks good.