Activity Log: All Times in UTC (PT)

15:58 (07:58) Chris – Beam tube sealing ~ 150 yards from End-X
17:40 (09:40) Peter – Transition LVEA to laser safe
17:51 (09:51) Bubba – In the LVEA working on crane rail shimming
17:52 (09:52) Peter – Reset PSL External Shutter flow sensor. 
17:55 (09:55) Keita – Testing on ETM-X
18:31 (10:31) Kyle – Going to Y-Arm Compressor room
18:43 (10:43) Reset tripped BS Stage 1 & Stage 2 WDs 
18:45 (10:45) Mitch – Working in the Optics Lab
19:16 (11:16) Betsy – Going onto LVEA to look for cables
19:21 (11:21) Mitch – Out of Optics Lab  
19:22 (11:22) Betsy – Out of LVEA
19:49 (11:49) Reset tripped ETMX Stage 1 & Stage 2 WDs
19:50 (11:50) Kyle – Back from Y-Arm
21:38 (13:38) Kyle – Going back to Y-Arm Compressor room
21:54 (13:54) Mitch – Going into the Optics Lab
21:55 (13:55) Christina – Forklift package from LSB to VPW
22:08 (14:08) Christina – Finished and forklift is parked back at the LSB
22:15 (14:15) Kiwamu – Opened main shutter (with Peter K approval) – LVEA is still laser safe
22:27 (14:27) Alastair & Nutsinee – Going into the LVEA  
22:34 (14:34) Mitch – Out of Optics lab
22:53 (14:53) Bubba & John – In the LVEA working on the main crane shimming
22:59 (14:59) Reset tripped OMC SUS WD.
23:40 (15:40) Alastair – Going into LVEA to turn on CO2 laser
00:00 (16:00) Turn over to Travis


End of Shift Summary:

Title:  02/04/2015, Day Shift 16:00 – 00:00 (08:00 – 16:00) All times in UTC (PT)

Support: Kiwamu, Hugh,               
 
Incoming Operator: Travis

Shift Detail Summary:  IFO was not locked during the day due to elevated seismic and microseism levels. Various groups took advantage of the no locking to do crane maintenance in the LVEA and make several fixes/upgrades. Ongoing commissioning work and testing during the shift.

FRS 4296, 4333
Bugzilla 969

The gds tools are now version gds-2.17.1.3-1, which should fix manual channel name input problems in DTT.  

When the "Start" button is pressed to run an analysis, DTT checks active channel names to see if they exist in the channel list.  If any do not, a popup window will appear to inform you of the channel names that could not be found.  You then have the option to cancel the test or continue with the channels which are valid.

This release is meant to allow users to copy/paste or manually enter channel names for analysis.  Since a manual entry does not provide any channel rate data, it is still possible to get the wrong channel using NDS2 when there are more than one channel rate to choose from.  

Kiwamu, Jim, Dave:

New h1lsc and h1omc models were created and installed this morning. The DAQ was then restarted.

The change was to expand a single filter module to two in series (providing the potential for 20 filters) for the DARM, MICH, PRCL and SRCL paths.

models were chaged at 11:11 PST and the DAQ at 11:13 PST.

Changes attached.

Updated userapps: isi/common/guardian/isiguardianlib/watchdog:

hugh.radkins@opsws1:watchdog 0$ svn up
U    states.py
Updated to revision 12546.
 

guardctrl restart ISI_ETMX_ST2:  this caused the guardian to deisolate stage2.  Not what I experienced Tuesday when I did similar..  When I restarted stage1 guardian, it did not touch the platform, as I expected.

Tripped the platform and now as expected, the damping loops were turned off when a state4 trip occurs.  Okay, so update fixed issue.

At ETMY, restart Stage1 first, then Stage2 and Guardian did nothing to the platforms...

Restarted HAM3, no problem.  Tested (tripping PR2 in the process) and the DAMPing path was turned off.  So good.

When restarting ISI_HAM6, this ISI guardian switched its request state to NONE but did not do anything, chamber manager was throwing messages.  Toggled ISI to HIGH_ISOLATED and everyone became happy.

TCS crew is working on SRC so I'll not touch them.

Guardian restart & test ETMX, HAM3.

Guardian restart ETMY, HAM2, HAM6.

I'll complete the remainder of the restarts and maybe test a couple more when TCS is clear.

Turns out that HWS at the end stations were temporarily turned on. When HWS is on and not on a separate power supply, it injects 57Hz noise into the ESD, which is picked up by the monitor. Another mystery solved.

On the attached top left, red is with the EX HWS off, blue is on, IFO unlocked, ETMs uncontrolled. EX ESD UL digital output was turned off. The 57Hz line and its harmonics are gone with the HWS.

On the bottom left are the same thing but with EY HWS off/on. No 57Hz in either of the two traces. That's because EY HWS is on a separate temporary power unit (alog 18355). The reason why we also saw 57Hz in EY LVESDAMON in lock (alog 25310) is not clear, maybe EX ESD was not turned off (though all outputs were zeroed) and shook the ETMX, and the line might have propagated to EY through DARM loop (we can see the line in DARM).

I don't know if X-Y difference (bottom right), apart from the callines in Y, is for real or just the sensing noise.

WP#5722

While we have high useism I wanted to revisit how this node is doing since I haven't touched it since before ER8. I created, started the node, and then had it switch blends with no problem. I then tried to mess it up to see how it would react. This is where I left off last time.

Some of my tests:

• If I manually change the blend to one that was not set by the GRD node, then it will complain that it is not in the right configuration and then try to change it immediately. At this point the FE has control of the filter banks so the node cannot actually change the value of the banks. I will need to change the code to wait before trying to change it back, or maybe just jump to DOWN and notify that it isn't in the preferred configuration.
• The DOWN state works as it should. It will jump to it from any of the requestable states and then just sit there and not do anything. If a blend is manually changed while the node is in DOWN, it will simply just ignore it. The node is, well, down.
• INIT doesn't jump to the state that it should, I'm not 100% sure why yet, I have some guesses but need to investigate some more first. If INIT is selected, even if it is in a state that it just set, it will go to DOWN and then work its way throught the graph to the requested state.
• Stopping a node in the middle of the switch still allows the FE to finish the "blending", but the NXT banks do not get turned off at the end. I need to think of a way around this, though this situation shouldn't ever happen...(I'm sure someone will find a way)
• Chaning the name of the filter in the const.py, but with an incorrect filter bank number will cause it to change blends to the FB number but as soon as it is done it will complain that it is not in the right configuration and then try to get to the state again. I should add a quick check somewhere that will check that the name and FB number match.

Overall it went well and I have a few things to work on. When I get another chance to test this out I will try it on ETMX with the BRS SC as well, the BLENSCOR node as I have been calling it.

I flushed the chiller water for both X and Y arms today, with advice from Jeff on what to do.  The water in the LHO chillers has not been a problem so far, and shows no evidence of particulate or discoloration.  However the chiller manual suggests that replacing the water every 6months would be a reasonable maintenance schedule.

Given that there was no problem with the water we could have got away with just draining and refilling the chillers a couple of times to replace the water.  However at LLO we will want to actively flush the system, so for that reason I also flushed the chillers here.

First I turned off the lasers, then their power supplies and the power supply to the AOM driver.  The laser, laser driver, AOM head and AOM driver are all water cooled.  For each chiller in turn I performed the flush as follows:

*Tools required - flat blade screwdriver for hose clamp

1) Turn off the chiller then disconnect the chiller using the quick connects, then drain the chiller using the drain plug at the bottom on the back.***Suggest making sure there are no power cables around on the ground at this point becase water may get on ground***

2) Close drain plug, refill chiller with distilled water ("laboratory grade" = reverse osmosis filtered then distilled).  Reconnect cables and the "Process Outlet" pipe to the top of back of chiller.

3) Leave the process inlet pipe disconnected and remove the quick disconnect attachment from the pipe.  Run this pipe into a water collection basin (I used a mop bucket which has wheels and is a good size for this)

4) Note that when filling the water collection basin, make sure that it doesn't get so full that you can't move it back down the stairs.  5 gallons is about as much as you would want to carry.

5) Repeat the following steps until all water is replaced

     a. Turn on chiller while holding pipe into basin

     b. Wait for approx 10 seconds of water to come out of chiller, then turn chiller off using the switch on the back (not the one on the front which takes too long to turn chiller off)

     c. Top chiller back up to full with distilled water

6) Once the water is replaced reconnect the process inlet pipe and hose clamp

7) Turn on the chiller and run for several minutes.  Check water level and top up as necessary.

I have measured the HARD arm loops at several different input powers, so that we can see how the peaks move around with radiation pressure. Some of the measurements could be a bit better (eg. CHARD YAW 10W to lower freq), but I think it's enough that we get the idea of what's going on, and can confirm with the ASC model. 

For each plot, red is 20W, orange is 10W and blue is 2W.  Shaded regions indicate the ~1 sigma error bars from the coherence of the measurement.  The gains and control filters are different for each of these measurements, but I know what the state was, and the model script takes those differences into account.  I will also divide out the rest of the loop contributions, so that we can see how the suspension changes by itself.

For all of the loops except for DHARD Pitch, we have basically no gain in the frequency region that I had the patience to measure. We should look forward to using the confirmed ASC models to design sensible loops that have some actual gain.

The DHARD Yaw comparison with the model was reported in alog 25054, but I have not yet made the comparison plots for the other degrees of freedom.  The oplev damping is still being added to the quad suspension model, so it may be another little while before the pitch measurements get a model to compare with.

Attached as a tarball is a folder including the DTT templates for these measurements, the transfer functions and coherence text files exported from DTT, the matlab script to create the figures, and all of the figures saved in several different formats.  Note that the coherences that are exported into the text files are just the IN2/IN1 coherence, although others are available in the DTT templates.

In bring the diagnostic breadboard back online, which required powering up the electronics,
the flow sensor sensor tripped causing the shutter to close.  It was reset from the Control
Room.

Back to normal.

The reduction in gain on chiller loop looks to have got rid of the oscillation that we were seeing.  Attached is the over night data, which shows the same small power output range and pzt range, with a smoother chiller response.  Y-arm laser wasn't on last night.

   Transition Summary:
02/04/2016, Day Shift 16:00 – 00:00 (08:00 – 16:00) All times in UTC (PT)
	
State of H1: IFO unlocked. The wind is a light breeze (< 7mph). Seismic and microseism are still as elevated as they were last night. Locking may be somewhat difficult at this time.      

It has been a fight to get DRMI to stay locked for more than a few minutes due to high ground motion.  Commissioners suggest calling it a night as microseism continues to trend upward.

The ground motion is just too high, so Evan, Travis and I agree that we should call it a night.  We can't hold DRMI for more than 2 minutes or so.  Earlier today, I flipped the phase of the ASAIR 90 PD, so that we get roughly the same number of counts when DRMI is locked as we used to.  We only use ASAIR90 for thresholding engagement of the DRMI ASC, so it's not a super critical PD phasing-wise.  I'll come back to it tomorrow.

This is another instance of getting the '75% notification', followed shortly by a WD trip on HAM6.  All I did was to reset the WD.

Vern, Jim, Dave:

Several times in the past few weeks we have seen an invocation of tconvert produce the output:

tconvert NOTICE: Leap-second info in /ligo/apps/linux-x86_64/ligotools/config/public/tcleaps.txt is no longer certain to be valid; We got valid information from the web, but were unable to update the local cache file: No permission to write in directory /ligo/apps/linux-x86_64/ligotools/config/public
 

This afternoon we spent some time tracking this down. tconvert uses a web server in france to access the IERS bulletin-C information page which alerts us to upcoming leap seconds. It appears that this server was pure HTTP until January 11th this year. It is now an HTTP server which redirects the request to a HTTPS page. The TCL code inside of tconvert interprets the redirection return message as an error. It then marks the expiration of the leap second information for 24 hours in the future.

The procedure we used to maintain future leap second notification was to twice a year run tconvert as user controls, which updated a leap seoncds database file (owned by controls) which was good for the next six months. With the recent HTTP issues, controls now produces a file which is good for only a day.

This afternoon we ran a modified tconvert which access a different leap seconds HTTP page and the LHO CDS database file is now good through 8th August. In the mean time we will raise an FRS and work with Peter S for a long term solution.

There seem to be two new rf oddities that appeared after maintenance today:

• AS90 seems to jump back and forth between two different offsets, maybe about once every hour or so.
• POP90 and AS90 both show irregular low-frequency motion, similar to what we see when an in-loop OSEM (or its sensing chain) goes bad. Ease of locking doesn't really seem to be affected, unlike the previous experience with bad OSEM electronics.

Attached are spectra comparing the SUS ETM L3 LV ESD channels from lock segments before (Jan 24, 2016 03:35:00 UTC) and after (Feb 1, 2016 01:50:00 UTC) the ESD Driver update last Tuesday alog 25175 (and the subsequent ESD fixes on Wed 25204).  Before the install, the channels looked like ADC noise, while they look live now.  The ETMx plot is included, but of course the ETMx ESD is not ON during either lock stretch, so all the plot really says is that something happened to the channels.  Whether or not the ETMy channels look like they actually should, according to various ECRs, is to be determined.