0915 - 0930 hrs. local -> To and from Y-mid Opened exhaust check valve bypass valve. Opened LLCV bypass valve 1/2 turn -> LN2 at exhaust after 28 seconds -> Returned valves to as found state. Next overfill to be Monday, June 13th.
SEI: BRS upgrade at End-X underway and going well. SUS: RMS WD at End-X is tripped and cannot be reset. Being worked on. VAC: Installing RGAs at both mid stations. All other groups report no outstanding issues.
Sheila, Evan, Carl Terra
At the end of the night we had several locklosses when trying to engage the soft loops. It seems like the problem could be turning on the 10 dB more offloading gain in the top mass, which we have been doing without a problem for several weeks now. In the last lockloss like this, L2 rms watchdogs tripped on all 4 test masses (not all coils). Evan manually reset the Y arm optics, but we couldn't reset the X arm optics. For ETMX, Terra and I went to the end station and tried power cycling the coil driver, but this didn't allow us to reset the watchdogs. Carl and Evan power cycled the ITMY coil driver, and were able to reset the RMS WDs but the software WD cannot be reset.
Also, we have had several EQs tonight (both large and small) and tried out turning sensor correction on and off.
TITLE: 06/10 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Commissioning
INCOMING OPERATOR: None
SHIFT SUMMARY: H1 was locking fairly well until the commissioners left for their dinner break. Then we were slammed by a series of EQs (6.1 in Nicaragua and 3 aftershocks and 5.9 in Solomon Islands). Since then, things have not been locking so well.
LOG:
3:46 Sensor Correction turned off in anticipation of incoming EQ.
3:57 ITMx and ITMy ISI WD tripped.
4:35 Attempted to start locking again, but could not get past DRMI. Realized this was due to another incoming EQ. Set IFO to DOWN.
5:58 Started Initial Alignment
6:35 Finished IA, back to locking
Bounce damping setting was changed for IY and EX so they damp at all. Don't know why the old settings don't work.
For EX I just flipped the sign.
For IY, I rotated the phase by 210 degrees in effect and increased the gain significantly. Gain increase might be unnecessary but I leave it like that because it seems to work faster.
| Old | NEW | |
| EX |
Gain= -0.03 (negative), FM1 (+60dg), FM4 (bp) |
Gain=+0.03, FM1 (+60dg), FM4 (bp) |
| IY |
Gain= -0.03 (negative), FM1 (+60dg), FM3 (BP), FM6 (+30dg) negative sign + 60 deg + 30deg = -90 deg total |
Gain= -0.15 (negative), FM2 (-60dg), FM3 (BP) negative sign -60 deg = +120 deg total |
ISC_Lock was changed but not loaded.
Travis and I added a filter with negative gain to IY bounce (and EX) damping, so the gains for all bounce and roll damping filters will be positive from now on. The guardian is updated for this.
We found that in the state DARM offset, we got good damping for ETMX with a positive gain and -60 degrees of phase. This worked better than the settings that Keita described in the DARM offset state, but we also used Keita's settings sucsesfully in DHARD WFS.
TJ and I saw that with the Keita settings at DARM_OFFSET we were ringing up the bounce mode for ETMX. So, now in the Resonance state the bounce damping gets set to the Sheila settings. Unclear why this one needs a 120 deg phase change but no others do. Perhaps related to why the settings needed changing in the first place? When the bounce damping is first turned on in DHARD_WFS, it goes back to the Keita settings.
We have been using a bandpass filter for ETMX bounce mode damping that is designed for the ETMY bounce frequency. For the other optics we use a broad bandpass, so I've now set the gaurdian to do that for ETMX as well. The new settings are:
FM3 (broad BP), and FM8 (gain -1). These are in the guardian for now, to be used in all states, so I've commented out the filter change in RESONANCE.
When we have the notches in the ASC off, we can damp the bounce modes as seen in the DARM spectrum. However, even when they appear to be reduced in the DARM spectrum they seem to be increasing in the MICH ASC signals.
I copied quad bounce and roll notches to the BS ASC loops as well.
I looked at a 1 mHz spectrum of the bounce modes from last nights lock, and the frequencies are consistent with what we reported one year ago:
| ETMY | 9.731 Hz |
| ETMX | 9.776 Hz |
| ITMY | 9.831 Hz |
| ITMX | not rung up last night but should be 9.847 Hz |
The confusion about the ITMY mode frequency (corrected by Evan in the comment) was unfortunately propagated to our monitoring filters. I've corrected the monitor bandpass for ITMY and made all of the bandpasses 10 mHz wide (before they were all 30 mHz wide, meaning that the monitors could not distinguish between the ITMs).
Sheila and I rewrote the analog CARM transition so that we now hop directly from the digital normalized in-air REFL signal to the analog in-vacuum REFL signal (rather than the analog in-air signal). This has worked twice in a row, so maybe this will help us avoid some of the CARM switching instabilities.
Title: 06/09/2016, Day Shift 15:00 – 23:00 (08:00 – 16:00) All times in UTC (PT) State of H1: IFO unlocked. Half day of maintenance. Commissioning: Recovering the IFO and working on problems Outgoing Operator: None Activity Log: All Times in UTC (PT) 14:45 (07:45) Ken – Working in Upstairs CER 14:47 (07:47) Peter – Taking survey of IO tables in LVEA 14:48 (07:48) Tim Nelson on site – to see Bubba 15:01 (08:01) Krishna & Mike – Going to End-X to work on BRS 15:21 (08:21) Gerardo – Going to End-Y Compressor room to work on IP11 wiring 16:04 (09:04) Bubba & Tim – Going into LVEA for facilities inspection 16:05 (09:05) Gerardo – Back from End-Y 16:07 (09:07) Electrical parts delivery for Richard – Dropped off at the LVEA rollup door 16:15 (09:15) Krishna & Mike – Back from End-X 16:16 (09:16) Bubba & Tim – Finished in the LVEA 16:17 (09:17) Kyle – Going to Mid-X to take measurements 16:18 (09:18) Filiberto & Gerardo – Going to End-X to work on Binary IO (WP #5925) 16:25 (09:25) Bubba & Tim – Going to End-X for facilities inspection 16:30 (09:30) Jason & Ed – Going into the PSL enclosure to work on DBB alignment 16:54 (09:54) TJ & Jim – Starting new Guardian nodes for both end stations (WP #5907) 16:56 (09:56) Kyle – Back from Mid-X 17:00 (10:00) Filiberto & Gerardo – Back from End-X 17:20 (10:20) Christina – Forklifting wood creates from mechanical building to staging building 17:27 (10:27) Bubba & Tim – Finished at End-X – Going to End-Y 17:38 (10:38) Haocun – Taking a tour through the LVEA 17:50 (10:50) Kyle – Going into the LVEA to get a helium tank 17:58 (10:58) Bubba & Tim – Finished at End-Y 18:00 (11:00) Haocun – Out of LVEA 18:15 (11:15) Sheila, Haocun, & Filiberto going to Mid-Y to get some boards 18:27 (11:27) Bubba & Tim – Going into the CER for facilities inspection 18:34 (11:34) TJ & Jim – Finished with Guardian work 18:35 (11:35) Jason & Ed – Out of PSL 18:40 (11:40) Karen – Cleaning in the H2 building 19:00 (12:00) Sheila, Haocun, & Filiberto – Back from Mid-Y 19:05 (12:05) Karen – Out of the H2 building 19:14 (12:14) Cheryl – Going into Squeezer Bay to look for parts 19:49 (12:49) Carlos – Going into MSR to take inventory 19:51 (12:51) Cheryl – Out of the LVEA 20:00 (13:00) Start alignment and locking after maintenance window 20:09 (13:09) Dave – Restarting all PI models and the DAQ 23:00 (16:00) Turn over to Travis End of Shift Summary: Title: 06/09/2016, Day Shift 15:00 – 23:00 (08:00 – 16:00) All times in UTC (PT) Support: Evan, Sheila Incoming Operator: Travis Shift Detail Summary: Adjust Y-Arm Fiber polarization from 16.5% to 6.0%. After half plus maintenance day, working on recovering the IFO. Considerable work was needed to get the IFO back into a functional alignment. That completed, the IFO is locking up to ENGAGE_SOFT_LOOPS. There is still a problem getting through REFL_IN_VACUO. Sheila and Evan are working on this problem.
The wires that were off the connector for the IP11 signal cable were fixed today, IP11 is back on and running without problems.
I zoomed in on where the oplev signal was before the outage, and where it is now.
While I was checking through the list of running nodes, I noticed that there were a total of 5 TCS nodes. It seems that the old TCS_ITMX diagnostic node was still running, but no one knew since it was not on the overview or anywhere else. Since the newer TCS_ITMX_CO2 node has replaced this one, I destroyed the node after confirming with Nutsinee.
DAQ EDCU was briefly GREEN until this node was removed, now back to PURPLE.
Added new SEI configuration Guardian nodes to all of the corner station chambers except HAM1 (soon). For the CS BSCs the code is almost identical to what is ran at the end stations, but with the BRS checks removed. The HAM config nodes are a bit different, as they do not change the blends, only sensor correction.
In the near future there will be a SEI configuration manager that can change all of the chamber configurations at once. This will let us prep the IFO for incoming earthquakes or change the configuration for different environments.
Updated Guardian Overview shot attached.
WP#5907
The hourly cronjob which keeps the nds jobs directory from overfilling was started on h1fs0 (moved from h1boot). Details at:
https://lhocds.ligo-wa.caltech.edu/wiki/CleanupNdsJobsDirectory
Tega, Dave:
New code for h1omcpi, h1susitmpi, h1susetmxpi and h1susetmypi. Slow channels were modified, therefore a DAQ restart was needed.
The DAQ restart also caught the recent changes to Guardian (new nodes) and HWS (removed heartbeat from ETMY).
The DAQ EDCU is now GREEN again.
Attached is a mdoescan of the DBB HPO path after some alignment attempts were made. It seems that TEM01/10 modes seems to be within reasonable values. Vertical alignment into the PMC seems to be slightly better than reference. Horizontal was marginally worse yet locking of the PMC was still not possible.
This scan was taken so we could have an idea of where we are after today's alignment work. Based on what I see, the HPO path of the DBB looks pretty good. Higher order mode power of 6.9% is reasonable, TEM01 (vertical alignment indicator) looks nice and low, although the TEM10 (horizontal alignment indicator) looks a little high, but not too bad. The mode matching peaks (the 2 larger peaks left and right of center) both look good when compared to the reference. All in all I think the HPO alignment into the DBB is pretty good. That said, as Ed says above, we are still unable to lock the DBB PMC. At this point I'm not sure what/where the problem is. Based on this mode scan I don't think it's an alignment/mode matching issue. Will have to think on this one...
No work was done on the Front End (FE) path today, will work on that once the HPO path is fully up and running.
Carl, Terra, Ross, Tega
Tonight we used the freshly installed LVLN ITMX ESD driver to ring up and damp two mechanical modes of ITMX, 15063 Hz and 15077 Hz.
After sorting out some phase settings, we drove the ITMX ESD close to saturation in a differential drumhead pattern. Negative gain rang up 15063 Hz. Flipping the gain sign to postive then damped this mode and rang up 15077 Hz. The amplitude plateaued as a the saturated drive signal approched a square wave. Figure below tracks amplitude of 15063 Hz and 15077 Hz (seen in OMC trans), with gain sign flip occuring around the 0.15 time mark.
Also attached is a spectrum of H1:OMC-PI_DCPD_64KHZ_A_DQ during no gain, negative gain, and positive gain times, i.e. on either side of the 0.15 time mark from the plot above.
At about 0.23 hours the gain was turned off and the mode rangdown. The fit to this ringdown indicates the mode has a Q factor of (omega_o)/(2alpha) = 1.2 million.
Settings: Power 1.9 W, DC bias 100k, butterworth BP filter, iWave bypassed, -60deg damp filter, damp gain 300,000.
Such a wonderful conclusion to the installation and commissioning of this system. Much thanks for the great support I received from all involved.
I suspect that 15077 Hz mode is an aliased 48923 Hz mechanical mode (64 - 15.077) kHz. The FEA gives an interesting mode at 48944 Hz (mode shape attached). Observation of the analog channel PSD on transmission is required to confirm if this is a case. The 15077 Hz mode is only 14 Hz above known 15063 Hz mode. I am not very familiar with linetracking filter but I assume that the two resonant lines 15063 HZ and 15077 Hz cannot be sufficiently separated and the signal with higher resonant peak will be eventuality phase-locked. It may be interesting to observe the transition from one mode to another. If this all is true you measured the Q-factor of one of the modes in the range were PI may also show up at higher circulating power than during O1. It looks to me that the 48923 Hz mode is very sensitive to off-center position of optical TEM00 (that's probably why can be seen on OMC) and can be used for centering of TEM00 on ITMs.
In the Q estimate 'f' was used in place of 'omega_o' introducing a 2 pi error. The corrected estimate of the Q factor of the 15077Hz mode is 7.4million.
Michael, Krishna
We made a lot of progress on the BRSX upgrades today.
The new belt driven turntable was installed which is designed to have less vibrations during damping. The previous turntable used gears and a loud stepper motor which caused vibrations which in turn drove up modes of the balance. (see 26265) The new turntable uses a belt and a much quieter Bekchoff stepper motor which has noticably less vibrations. Also, the Mightex camera, a 2048 pixel CCD read over USB, was replaced with a Basler camera which has 4096 pixels and is read over ethernet. This new camera will remove the USB communication error which was causing the system to crash every couple weeks.
Additionally, we replaced the laptop and NI DAQ card with a Beckhoff computer and electronics. Both the C# code, which turns images into angles, and the PLC code, which controls the damper and sends data out, are working and data is being sent out to the seismic front ends. However, the EPICS communication has yet to be set up so many of the diagnostic channels are not being read.
The balance is rung up to a very large amplitude due to the work today but we've begun to slowly damp it down with the new turntable.
Here are some pictures of the damper before and after the upgrade. The first one shows the old turn table in the bottom left under the arm and the Mightex camera wrapped in foil on the very top of the autocollimator. The second one show the old stepper motor and gears. The third shows the new turn table on the left, the new electronics box in front of the chamber, and the new Basler camera and mount on the top. The third is more detailed view of the turntable with the new stepper motor in the back and the drive belt running below the damping masses.
Also attached is a video showing the smooth motion of the damper. This was while the balance was rung up so the damper is moving from the counter-clockwise limit in beginning of the video to almost the clockwise limit about two thirds through.
Terra, Carl B., Rich Terra and Carl had noticed a ~20mVp-p 1.3MHz oscillation on the ETMX ESD Driver at viewed with an oscilloscope at the output (SHV connectors on the front panel) leading to the ETM. After confirming that the oscillation was really generated by the ESD Driver, Carl and I opened the spare to see if we could find a set of conditions that might produce such an oscillation. By examining the spare, we were able to trace the potential instability to the first stage of the two stage pole-zero dewhitening filter. Poking at this stage with my finger will occasionally get it to latch into the 1.3MHz oscillation state. We drove the output of the opamp in question with a 1vp-p, 1kHz square wave through 400 ohms to get a better feel for the dampling and phase margin under normal operating conditions. This stage is completely stable with the pole-zero stage engaged, but becomes marginally stable with the pole-zero stage bypassed. The problem can be cured by the addition of a small (~30pf) capacitor across the feedback 21kohm resistor implying it is worsened by input capacitance. We opted not to pull all the drivers and fix this problem. The reasoning is that this stage is in its stable state (pole-zero engaged) while the driver is used as a low noise driver for length control of the ETM. There is really no driving need to fix this at this point, but we wanted to document its existence for optional repair at a later date. A fault report will be written to further document the issue.
Added to FRS https://services.ligo-la.caltech.edu/FRS/show_bug.cgi?id=5675
Here is a time series of L2 master outs durring the lockloss where the RMS WDs tripped. This lockloss happened durring a 15 second sleep in the guardian, which is why down was not run for so long after the lock was lost. We've eliminated this particular long sleep but there are other places where we have similar sleeps that need to be fixed. We went through the guardian states up to DC_READOUT this afternoon, rewritting in places that have long sleeps. We've tested this up to ENGAGE_SOFT_LOOPS.
The version of ISC_LOCK committed just before starting these edits is 13570, the version after is 13576.