Title:  08/16/2016, Evening Shift 23:00 – 07:00 (16:00 – 00:00) All times in UTC (PT)
State of H1: Relocking the IFO after maintenance   
Commissioning: 
Outgoing Operator:  TJ
 
Activity Log: All Times in UTC (PT)

23:00 (16:00) Start of shift
23:05 (16:05) Jason – Out of the LVEA
23:38 (16:38) Jason – Going into the LVEA to turn on the TCS-X laser
23:43 (16:43) Jason – Out of LVEA – TCS-X laser has been restarted
00:05 (17:05) Rick S. and crew – Out of the LVEA
01:18 (18:18) HAM5 ISI and SRM WDs tripped. – Perhaps commissioning activities 
03:47 (20:47) Sheila & Terra – Going to ISCT1 to align camera
05:10 (22:10) Sheila & Terra – Out of the LVEA
05:15 (22:15) Jeff – Into the LVEA to check doors and run particle counts
05:25 (22:25) Jeff - Out of the LVEA

Title: 08/16/2016, Evening Shift 23:00 – 07:00 (16:00 – 07:00) All times in UTC (PT)
Support:  Sheila, Jenne, Terra 
Incoming Operator: N/A

Shift Detail Summary:  The commissioners spent the shift working stability and robustness of the IFO. 

   There were high particle counts recorded on both dust monitors (Beirgarten and at HAM6) and in the PSL Anti-Room starting at 09:30. The high counts were mostly in the 0.3um particle sizes. Elevated counts were not recorded in the PSL Laser room. Counts were tapering off by the start of the evening shift and by shifts end had returned to somewhat normal levels. Have not been able to determine the source, yet.   

Terra, Sheila

Tonight we had trouble engaging the ASC again.

• We still have trouble turning up the gain in DHARD, and didn't get a chance to look at this tonight. We did try reverting the ETMX notches but that doesn't solve the problem. 
• We also had trouble with our MICH error point being wrong, which might have been related to TCS being off durring maintence.  Now that TCS has been on for several hours the MICH loop seems to be working OK. 
• We had to restore OMC QPD offsets that weren't in SDF before today's boots.  We are now back in DC readout. 
• Once we reached DC readout we manually tried the CARM gain redistribution that we had commented out of the ANALOG_CARM state yesterday, and we did not see glitches reappear in the MCL drive.  We left this uncommented in the guardian, so we can try this in the morning.
• Jenne reset the soft offsets for our new alignment, and we also got rid of the POP A yaw offset.

Losing optical gain in POP X

We rang up what we think is a PR3 bounce mode when engaging the ASC the same way as last night.  We found that we could avoid ringing this mode up by keeping the PRC2 gain low (digital gains of -500).  Right before the OMC damage/vent, the POP X path was reworked and the optical gain seemed suspiciously low. 

Tonight we found that the optical gain has decreased even more.  Terra changed the demod phase by dithering PR3 pit (500 counts to M3) and rotated the phase positive 65 degrees, (Q1, Q2, Q3, Q4 from 55, 53, 54, 51 to 120, 118, 119, 116 ) to maximize the signal in I (minimize the Q signal).  The 2 attached figures show Terra's before and after OLG measurements (excitation gain of 50), both with Jenne's gain of -5000, showing a 10dB increase in optical gain which is about what we expected based on the dither amplitude change. 

After optimizing the phase, we did not see the 28 Hz mode get rung up, but this seems to come and go because we also didn't see it yesterday.  We quickly tried moving L2 on the POP X path, while watching the amplitude of the PR3 dither line in the POP X signal.  We moved the lens about 4 inches closer to POP X and about 3 inches further away, and didn't find any location that had more signal for PR3 so we replaced it as we found it. 

We are going to leave the IFO locked in DC readout 2 Watts with the request set to down so that it will not try to relock. The noise is bad as expected. 

PSL model changes. WP6084

Ben, Jim, Dave:

Three psl models were changed today:

h1ioppsl0: Had the new Contec 6464 binary IO card added.

h1pslpmc: Removed its grounding of the last four DAC channels, these channels will now be used by the new outerloop system

h1psliss: added the new outerloop test code for this week's chassis install and test by Ben. Current code just monitors the ADC input channels and writes them at 2kHz to the commissioning frame. I permits the four DAC channels to be driven by filter modules, it provides binary control and monitoring of the eight BIO channels.

I have created a simple MEDM and attached itto the site map (PSL -> 'ISS BEN TEST').

We anticipate more h1psliss code changes as this commissioning progresses during the week.

New GDS code

Jim:

please see Jim's alog.

Reboot h1guardian0

TJ, Dave:

h1guardian0 was rebooted after running 29 days. The loading came down a bit but starts off in the mid-20s.

New digital video camera

see today's alog for details

Wind gust meter attached to h1pemex

Jim W, Jim, Fil, Richard, Dave:

LHO's Young ultrasound 3d anemometer was attached to three spare ADC channels on h1pemex. The channels are ADC_0_12,13,14.

Frame Writer Code Testing

Jonathan:

New daqd code was tested on h1fw0. The new code has been left running.

Beckhoff C1-PLC1 code change

Daniel:

please see Daniel's alog

DAQ Restart

Daniel, Dave:

DAQ was restarted to sync to new h1psliss channels and the latest Beckhoff h1ecatc1plc1.

   After running a low battery shutdown test, dust monitor #7 has been removed from the LVEA. 

Isolated pump carts from HAM5/HAM6 annulus system but leaving carts running.  AIPs were started earlier in the day and pump currents look low enough as to not need assistance -> Will shut down 2 of 4 running pump carts in the morning if not needed.  Other pump carts will need to run until Friday morning judging from current data

TITLE: 08/16 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Aligning
INCOMING OPERATOR: Jeff
SHIFT SUMMARY: Pretty busy mintenance day. Much more than I expected at least. TCS crew is still working on diagnosing TCSY, commissioners are trying to lock without disturbing them too much.
LOG:

• 14:00 Chris C to both ends for work in the fan rooms.
• 15:08 Bubba to LVEA t ofind 3IFO parts
• 15:13 Ed M in and out of LVEA at SUSR3 (WP#6074)
• 15:35 Christina & Karen to ends
• 15:37 Richard to LVEA table camera work
• 15:49 Fil to LVEA t odo cabling over HAM1 and HAM4 (WP#6086)
• 15:50 Corey to LVEA for ISCT1 camera work (WP#6071)
• 16:02 John, Bubba to LVEA
• 16:12 Jason, Alastair to LVEA TCSY work (WP#6083)
• 16:16 Richard to LVEA to Corey
• 16:20 Guardian reboot
• 16:46 Hugh starting HEPI WP work in CS mech room (WP#5986)
• 17:01 Joe to LVEA
• 17:02 Gerardo to LVEA starting cable pulling (WP#6089)
• 17:05 Norco at the gate
• 17:05 Corey out
• 17:12 Vern, Alastair, Jason to LVEA looking at TCS inventory
• 17:14 Joe out
• 17:52 PSL FE restart (WP#6084)
• 17:56 Joe to all out buildings
• 18:00 Marc to Ends for CPS timing update (WP#6092)
• 18:09 Rich, CJ to LVEA for PCal ITMX pictures (WP#6078)
• 18:38 Jim W, Ian to EX for annemometer work
• 18:45 Hugh to end stations for HEPI maintenance
• 18:51 Marc back from ends
• 18:57 Joe back
• 19:05 Jim W, Ian back
• 19:13 Jason, Alastair out for lunch
• 19:20 Hugh back from end stations
• 19:30 Gerardo out
• 19:47 Mike and guest into LVEA for tour
• 19:53 Richard to LVEA to Fil
• 19:54 Ben Abbot to LVEA to PSL rack
• 19:56 Richard out
• 20:10 Rick, CJ to LVEA to PCal ITMX
• 20:12 Fil out
• 20:38 PSL tripped off
• 20:46 Mike, guest out
• 21:09 Fil to CER (WP#6034)
• 21:15 Dave, Jim back from EX
• 21:24 Ben to LVEA to PSL racks
• 21:43 Jason back out to TCSY
• 22:24 Kyle to LVEA to get temps for vertex RGA
• 23:02 Fil back out to check on magnetometer near HAM2

PEM power supplies in the CER mezzanine were swapped out for Kepco model JQE supplies. This model is a 1/4 rack version, and allows for four power supplies to be mounted on one shelf. This is to make room for the ±24V power supplies that will be used to power Beckhoff/Baffle PD Amplifier/ Spool X/Y camera ect. All PEM instrumentation microphones, magnotometers, ect. were powered down from around 10:30 am to 1:00 pm. All power is now restored.

Cabling for the readbacks for the TCS RF Amp was pulled in this morning. Readbacks are connected to the slow controls concentrator in rack ISC-C3 slot U33/32, port 3. This closes out fault report 6025.

The PSL power watchdogs tripped at 20:38:13 UTC (13:38:13 PDT).  The cause appears to be the NPRO turning off.  In the attached trend the NPROOK channel, which monitors if the NPRO is working or not. goes to zero (indicating the NPRO turned off) at 20:38:08 UTC, 5 seconds before the power watchdogs trip.  The cause of the NPRO turning off is currently unknown.  I am bringing the PSL back up.

The most recent svn software version is running now on all Beckhoff computers. Changes In detail:

• Support for Twincat 3 in tcioc (should have no effect).
• AS protection: Added reset button for test and lock loss statistics.
• AS protection: The high power locking flag now requires the trigger photodiode to exceed the high level for a minimum of 1 second.
• AS protection: Aborting a test now causes the fault flag to show "Aborted".
• AS protection: Code interface documentation at E1600257.
• WFS demod: Fixed LO power calculation.
• tcioc startup: Added a 2 sec delay in End-X to prevent the tcioc to start running before TwinCAT is ready.
• Corner PLC3: Allocated more memory for constants, since we were running short.

All computers have been booted to make sure we are running a clean version.

Jim W. noted that a bleeding off of these accumulated saturation counts should be added to the HEPI models and this task depreciated. Only HAM1 had saturation counts to reset (it is not on the HEPI L4C WD SAT Monitor MEDM screen). Jim W. reset it and I did not see the number of counts.

Re WP 5986.  Currently just running three of the four pumps at the Corner Station as I have one off for cleaning.  This pump has been running from the beginning with little break and I was not always so fastidious about cleaning up the drips.  It was an mess and I think there are are a couple bearing leaks at the motor and the pump.  Being cleaned up now though will help evaluate if further action/worry is warranted.  With just three pumps running the drive out to the pumps is about 30% higher than before--may show in a periodic famis task if it is executed during this period.

It's ready to go back together and essentially it is together but some of the fasteners are just impossible to get to without disconnecting plumbing (messy.)  Going to shop for or fabricate a couple custom tools to help reassembly.  WP remains open.

(M. Pirello)

Per Richard's Work Permit #6092 I reprogrammed both CPS fanouts in the LVEA, and the ones on the X and Y end stations.  Programming was successful and the heartbeat LED's are now off.

I have added a check to the LocklossShutterCheck guardian that will add to the guardian log a note whether the HAM6 GS13s saw a large kick or not.  Note that this does not in any way change the functionality of this guardian, in that it still requires an Operator to run and look at the plot, then manually reset this guardian. 

However, we should be able to look through the guardian log to ensure that each time we lose lock, the log notes that the GS13s saw a big kick.  When we are satisfied that this is a good measure of the shutter having closed at the last lockloss, we can consider using it as an automation metric in lieu of Operators manually resetting the guardian. 

I have set the threshold for "big kick" for the GS13 to be 30,000 nm/s (1nm/s / count, so 3e4 counts).  When the shutter closes fast, the GS13 sees a signal as large as 90,000 nm/s, but when the shutter closes slowly it only kicks the GS13s to about 10,000 nm/s.  The threshold of 30,000 nm/s should differentiate between a fast vs. slow closing of the shutter.  Normal GS13 signal is under 100 nm/s peak. 

Attached is the code, which is also checked into the svn.  The idea is (once we're ready to make this automated) that if we see the GS13 kick, the CheckShutter state will automatically go to the LowArmPower state, and allow the IFO to relock.  If there is any problem, including inability to fetch the data, the guardian will stay in the CheckShutter state, and will require human intervention.  One could also consider adding light level checks to this, but I think that's probably overkill given the other checks that we already have in place.

Sheila Evan Jenne Terra

The IMC length drive has large glitches when only the IMC is locked.  The first attachment shows the MC2 M2 length drive durring a quiet time on July 31st, and a glitchy time today.  There are glitches present in both traces, but they are much louder in the trace from today.  If you look back at data from late July, the glitches have intermittently been present even before we started having trouble locking, so this is probably not the main reason we can't lock. 

The second attachment shows 15 minutes around the time of the July 31st trace, when the glitches transition from being quiet to loud. 

Short version: The new shutter seems to be closing harder than before the vent. This is causing the ISI to trip. So far we have gotten around this by running the ISI in a low gain state, but this makes the table motion much worse above 1 hz.

After the shutter work, we started having problems with trips of the HAM6 ISI when the new shutter trips. This morning we put the ISI in the "Robust" isolated state, which uses low gain isolation loops with UGFs around 10hz. The normal loops have UGFs around 30-40 hz. For now this seems to have eliminated the tripping of the ISI, but the performance of the ISI is much reduced above 1 hz. First plot shows a comparison Y GS13s and ground STS for HAM6 for the two configurations. Red traces are with the high gain loops, blue traces are with the low gain loops. Below 1 hz they look roughly the same (low frequency diffs seem to be mostly in the ground for blue), but at 1 hz the high gain loop is doing ~100x better. Not surprising.

I've looked the GS13s in Z at shutter triggers before and after the vent and the shutter seems to be moving the table a lot more now. Attached plot shows a trigger from today with the 10hz loops (blue), yesterday with 30hz loops ( orange/ spiced cider) and Aug 1st with 30hz loops (red). I've looked over at least 3 events in each configuration and they look remarkably consistent, with respect to ISI configuration. Shutter triggers before the vent seem to have kicked the table less than they do now. Maybe this will reduce as the shutter breaks in?

We have some medium gain loops installed with slightly less gain that the high gain loops, tomorrow during maintenance I'll try testing those, but I doubt they'll be enough. Otherwise, we can come up with some loops with some better compromises between UGF and high frequency roll-off than the low gain loops. But the best option would be to figure out  what can be done to reduce the kick from the shutter.