The attached plot contains time series of the kappas from about 2 - 6 UTC on June 24. Note that the GDS curve (which used TX PD) for kappa_{tst}, kappa_{pu}, and kappa_c remains stable, while the CALCS curve (which used RX PD) shows an oscillation likely caused by temperature dependent pcal clipping.
For comparison to a longer stretch of data, see the summary pages:
https://ldas-jobs.ligo-wa.caltech.edu/~detchar/summary/day/20170624/cal/time_varying_factors/
Also, note the correlation with temperature as seen on the bottom right plot here:
https://ldas-jobs.ligo-wa.caltech.edu/~detchar/summary/day/20170624/cal/pcal_y/
A similar daily trend can be seen for the past week or so.
at approx 8:27UTC H1 lost lock. There were no indicators on any of the FOMs to indicate impending lock doom. Sheila has just logged on remotely as reported by the freshly restarted Verbal Alarms which did NOT report the lockloss when it happened. Now it seems that Verbal Alarms are one step behind Guardian or not reporting anything at all. DIAG_MAIN is reporting that ESD Y driver is off, but I can't remember if this is normal for this part of the lock or not.
Also, on a side note, PT424 and 524 B are in the red.
Re_Locking:
First attempt - Lost lock somewhere around PREP_TR_CARM. Verbals are no help. Restarted again.
Second attempt - going better. Verbals still dodgy/sporadic. So, I was prompted by guardian to "HELP ME!!" due to the fact that we had reached LOWNOISE_ESD_ETMY and the ESD driver was still off. I turnd it on by clicking the HV ON/OFF button on te ETMY MEDM screen. Now guardian seems to be stalled so I will attempt to move it forward manually. That worked. And it broke at Noise tunings. Shades of what I walked in on at the beginning of my shift. Will try a third attempt.
Third attempt - I believe it broke at SWITCH_TO_QPDS. Restarted the computer with Verbals and AH to no advantage.
Fourth attempt - I stopped Guardian at SRM_ASC_HIGH_POWER to manually step through NOISE_TUNINGS. The step-through process seemed to work. I may have made my mistake at manually jumping over to CLOSE_BEAM_DIVERTERS instead of proceeding with the NOISE_TUNINGS step? I made i to NLN but SDF was showing some PSL ISS diffs that I don't know why they were there. I tried reverting them. That was a bad idea. Guess I should have accepted and moved on. Lesson learned. Back to square 1. :/
stay tuned...
Next attempt got me through ACCEPTING the aforementioned ISS diffs only to be left with a waiting for IMC_LOCK node notification (see screenshot). When I attempted to request the nominal state for this node everything came loose.
12:20UTC I'm leaving H1 in NLN but not Observing. I don't know how to get the IMC_LOCK node into it's nominal state without breaking the lock that I have.
Keita and I looked into one of these locklosses from last night, and for one of them the cause seems to be related to the ESD turning off (before the lockloss). Ed got the error message about the ESD in the next lock acquisition and reset the ESD.
The last couple of problems that Ed had are related to a problem with running the guardian by hand line by line. In the code there are places where ISC_LOCK makes a request from other guardians (in NOISE_TUNNINGS there is a line nodes[IMC_LOCK] = 'ISS_DC_COUPLED') this doesn't work in the command line unless you have initialized the node manager. The easiest way to do this is to request the state from the IMC guardian using the medm screen. In this case, the ISS has to be DC couped before the IMC boost comes on later in the state, doing these things in the opposite order breaks the lock.
We will continue investigating the problem in the NOISE_TUNNINGS that caused Travis and Ed to do it by hand in the first place.
TITLE: 06/27 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 75Mpc
OUTGOING OPERATOR: Travis
CURRENT ENVIRONMENT:
Wind: 4mph Gusts, 2mph 5min avg
Primary useism: 0.01 μm/s
Secondary useism: 0.12 μm/s
QUICK SUMMARY:
I walked in on Travis doing line-by-line Guardian at SRC_ASC_HIGH_POWER with Sheila on the phone. It seems whatever was going on was resolved before he left. While they were oin that I had to damp PI mode 28 which took some doing. It was starting to drag mode 20 and mode 27 with it and the PLL was not staying tuned very well so I was chasing that around for a while to get a more positive hold on the changes I was trying to effect with phase and gain. Those change diffs were accepted by Travis. At this time Guradian is reporting that PSL_ISS: diffracted power is high (6.7%) but DARM between 10 and 20Hz looks really good compared to last night and we're above 70Mpc in range. I'm not complaining.
Correction: Guardian was stopped at SRC_ASC_HIGH_POWER so he could step through NOISE_TUNINGS.
TITLE: 06/27 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing
INCOMING OPERATOR: Ed
SHIFT SUMMARY: After the thunderstorm knocked us out of lock, it has been a struggle to relock. After losing lock multiple time at CARM_ON_TR, I consulted with Keita. We walked through the state line by line and made it successfully through, but then lost it again at SWITCH_TO_QPDS. I decided to try an initial alignment to make sure it wasn't alignment that was hurting us. After IA, I was able to make it past the previous problem areas and consistently to NOISE_TUNINGS where it again would suddenly lost lock. I contacted Sheila and we walked through the NOISE_TUNINGS state line by line. This time there were no issues and we made it to NLN. I think the IFO just wanted a little attention.
LOG:
Accepted SDF diff in screenshot.
The storm has passed and I am attempting to relock. Struggling to get past CARM_ON_TR.
Almost definitely due to wind. We are having a pretty nasty thunderstorm with winds to 50+ mph.
I made a new SRCL FF filter, described in CSWG log 11217
While LLO was down briefly we had the chance to try it out and it seems like the performance is about 10dB better than the old feedforward in the band where we needed improvement, 20-50Hz. This is now on, in the guardian and accepted in SDF.
It seems to have reduced the srcl coherence, and perhaps had a tiny positive impact on the DARM noise. (3rd attachment)
For Sheila to load filter files for H1LSC. LLO was down.
TITLE: 06/26 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 63Mpc
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
Wind: 12mph Gusts, 8mph 5min avg
Primary useism: 0.04 μm/s
Secondary useism: 0.16 μm/s
QUICK SUMMARY: Observing for ~4 hours. No issues handed off.
TITLE: 06/26 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: OBSERVING at 65Mpc.
INCOMING OPERATOR: Travis
SHIFT SUMMARY:
Other than accidental lockloss, H1 has performed well. There was some commissioning time today.
Have a RED H1LSC on the CDS Overview related to Sheila's SRCL filter change (commissioning task). When we are out of OBSERVE, we should Load Coefficients for H1LSC & then Sheila would like to ENABLE the filter.
Winds have been picking up & we did get up to 100degF.
Addressed some SVN critical file changes (ops checksheet task).
LOG:
John, Bubba
Some LSB temperatures were in alarm and after investigating we found the chiller to be running on one of two circuits only. After resetting several circuit breakers the unit has started up and appears to be running normally again. The chilled water temperature had risen to ~65F and has now fallen back to ~46F. We are not sure what was wrong but one fan controller was off. This would have generated the "high condensor pressure" fault which was recorded.
Sheila, Jenne, Keita
Because of problems over the weekend with the CSOFT instability, we have engaged the ISS 3rd loop again. It will come on using the IMC lock guardian at the begining of the INCREASE_POWER state, and stay on in low noise. Turning it on when the ISS was already DC coupled caused our lockloss around 17:30 UTC.
Over the last week I have been slowly adjusting rolls offs and filters in ASC to make sure that ASC is not limiting our DARM noise above 20Hz. The first attached screenshot shows a before/after comparison of the largest arm control signals.
In the process I got loop measurements, which I am attaching here for reference although they were made before the changes.
The loops which I changed were:
CSOFT P: I rolled off the control filter earlier, swapping FM4 (cntrl2) for FM8 (cntrl)
DHARD P: I modified the roll off filter to be a little more aggressive. (FM1 ELP10b replaces FM4 ELP10)
DHARD Y: I changed the 25W compensation filter for one that gives us a little more phase near our ugfs and more gain at low frequencies. Then I made the roll off more aggressive. So FM2 (s25W) and 3 (ELP10) replace FM9 (25W) and FM1 (cutoff40W)
CHARD P:I changed the cut off filter a little. (ELP10)
These changes are in the guardian and have been accepted in SDF.
Check and adjust pressures and temperatures on the three dust monitor vacuum pumps. All within spec. Pump discharge filters show no increased dust accumulation.
Chandra, Dave:
Chandra requested that on the vacuum pressures web striptool the channel PT210B be replaced with PT243B, and PT310B be replaced with PT343B. This I have done, the old and new channel lists are shown in the attachment.
Attached are the VPW desiccant plot with this month's added data. Nothing out of the ordinary shown.
The dust monitor in the Diode Room is hung, (no response to remote commands). When into Diode Room and power cycled the unit. Ran zero count test and flow rate check. Both tests are within spec. After the power recycle of the dust monitor the unit is still hung up. Need to recycle the comptroller to recover remote communications. Fillberto will power cycle the comptroller during Tuesdays window.
Attached are spectra for HAM and BSC ISI CPS sensors.
The xterm window did not display any that were over threshold.