Everything looks normal. Evidence of current adjustments made to the HPO Diodes last week are the only thing of notice.
TITLE: 06/28 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 68Mpc
OUTGOING OPERATOR: Travis
CURRENT ENVIRONMENT:
Wind: 16mph Gusts, 12mph 5min avg
Primary useism: 0.04 μm/s
Secondary useism: 0.09 μm/s
QUICK SUMMARY:
TITLE: 06/28 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 68Mpc
INCOMING OPERATOR: Ed
SHIFT SUMMARY: Some trouble getting to NLN during the first half of the shift until Keita disabled the ISS 3rd loop. Since then, we have been in Observing for 3 hours. No other issues.
LOG: See previous aLogs.
I remembered that there is one step to disabling the 3rd loop that isn't in the guardian. We need to turn off the input to the third loop filter bank by hand before Noise Tunnings, so that it will not be turned on by the guardian as it is written now. I am not sure if Keita already did this and accepted it in SDF, but I forgot to mention it on the phone.
The annulus pressure in HAM 11,12 (diagonal volume) is rising and causing the both AIPs to fall out of range. Both AIPs were replaced in May 2016. Today I leak checked all door and flange outer o-rings on HAM 11, 12, and also AIP conflat flanges. Found e-5 Torr-L/s of He range leak at bottom of HAM 11 south door and e-7 Torr-L/s of He range leak on HAM 12 south door. Kyle tightened bolts but flanges were already metal on metal. Suspect there is an imperfection on outer o-ring seal. Will address next time we have the opportunity to vent (or decommission) the diagonal volume.
FRS 8411
Finally back to NLN and Observing.
Lockloss 2:43 UTC. Cause not obvious.
Back to Observing at 3:55 UTC.
After retuning the SRCL FF yesterday (37150), Travis eventually got the interferometer locked after some problems with a windstorm, the CARM offset reduction, and eventually the ISS second loop. That lock in the first hour or so of Ed's shift had a better range than we have had in quite a while.
The time for the better spectrum below is June 27th 2017 7:39:16 UTC, while the "before" time is June 26th 2017 19:25:21 UTC. It would be interesting to take a look at a long bruco report for the after time. The jitter is a little lower in the after time, which is probably just due to an alignment drift. The range at this time is the best we have had in a long time, as shown by the 90 day trend in the second screenshot.
Ran A2L while LLO was still down. After A2L completed, we saw a lot of noise in the DARM spectrum below 100 Hz. After a few minutes of the commissioners investigating this noise, we lost lock. Running IA now in hopes of a better lock next time.
TITLE: 06/27 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 63Mpc
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
Wind: 16mph Gusts, 12mph 5min avg
Primary useism: 0.04 μm/s
Secondary useism: 0.09 μm/s
QUICK SUMMARY: Just coming up to NLN as my shift started. So far, so good. No other issues handed off.
Observing at 23:07 UTC.
J. Kissel, D. Barker FRS Ticket 8218 Executive summary: Charge measurements complete after having "restoring" the "broken" infrastructure to do so. Charge looks OK, but not great. Details: After a few weeks of work, Dave was unable to identify any problems with the charge measurement scripts originally reported in LHO aLOG 36338. So, today, we just tried running the scripts again, and it worked. #facepalm Our best guess is that what error was reported was a bogus error, and the real problem was a result of NDS2 upgrades vs. Debian OS upgrades that were going at the time. It's a shame we didn't try at any other point during the interim weeks. Anyways, here are the latest charge measurements. Recall that we had tried some level of charge mitigation during the May 8th commissioning / vent period in hopes to reduce / turn around the ETMY accumulated effective bias voltage from ~40-50 [V]. We appear to have paused this accumulation, at least from what little data we have thus far. Another contributing factor may be that our duty cycle has been substantially less since the vent, so I imagine that we've had more time with the opposite signed bias voltage that also will have helped. Thus, I think we're essential picking up right where we left off before the vent, where it's on the borderline of being concerning, but not requiring immediate action just yet. Remember that this technique reports the angular actuation strength per quadrant, and not the longitudinal actuation strength. I normal post the longitudinal relative actuation strength via the calibration lines from the summary pages but - The summary pages are down, - We changed our computation of the time dependent correction factors to PCALY TXPD today, so the results may be skewed, and - We've had no new nominal low noise lock stretches since that change I'll attach a comment later that shows how the longitudinal actuation strength is doing. However, Aaron's plot in LHO aLOG 37158 shows that the accumulation is ~4% or so, again, we're in the same boat as before the vent. ETMX appears to have continued to keep a low effective bias voltage accumulation.
TITLE: 06/27 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: PREVENTATIVE MAINTENANCE (attempting to get to NOMINAL LOW NOISE!)
INCOMING OPERATOR: Travis
SHIFT SUMMARY:
Most Maintenance activities were completed around noon (19:00utc) today. Have dropped out at various points in the locking attempts this afternoon. We are wanting to see how H1 handles NOISE_TUNINGS via Guardian.
LOG:
09:44 PDT DAQ restart. Additional H0EDCU_VAC.ini channels for end station BRS annulus ion pumps. First install of the H0EDCU_SAFETY.ini file. Good start, no drama.
Jeff, Dave:
This morning during maintenance Jeff successfully ran the charge measurement script on ETMX and ETMY. I'll let him fill in the details, but this closes the LHO part of FRS8218. We suspect the core problem was an nds2 which has since been resolved.
The "red herring" SIStrLog error turned out to be a true error, fixed by opening up file permissions in the /ligo/logs/h1/inject_log area. I've opened FRS8405 to cover this.
I'm not sure this is common knowledge, but every time an awgstream runs it logs in a daily log file, for example
/ligo/logs/h1/injection_log/2017/06/27/injection.txt
...
1182627394.000000 H1:SUS-ETMY_L3_ESDOUTF_LL_EXC start zotws10 16523 1 awgstream ./test_sinewave_ETMY.txt 1
1182627418.059011 H1:SUS-ETMY_L3_ESDOUTF_LL_EXC stop zotws10 16523 1
1182627430.000000 H1:SUS-ETMY_L3_ESDOUTF_UR_EXC start zotws10 16536 1 awgstream ./test_sinewave_ETMY.txt 1
1182627454.055341 H1:SUS-ETMY_L3_ESDOUTF_UR_EXC stop zotws10 16536 1
1182627466.000000 H1:SUS-ETMY_L3_ESDOUTF_LR_EXC start zotws10 16548 1 awgstream ./test_sinewave_ETMY.txt 1
1182627490.054266 H1:SUS-ETMY_L3_ESDOUTF_LR_EXC stop zotws10 16548 1
Looked at the ~-4300 ct offset on the ETMY LL Fast I Mon channel. Cable was reseated at both the IO and AA chassis.
Offset dropped from ~-4300 cts to ~-250 cts. Noticed ADC interface cards on the IO chassis are missing the mounting hardware.
J. Kissel, F. Clara Talking to Fil after he got back from this week, he has indeed fixed the FAST IMON channels that were showing the ~4300 [ct] from one of the ADC legs of the given channels being shorted, but it looks like upon reseating the cable, the connector then shorted some of the NOISE MON legs. As the original documentation points out (see LLO aLOG 1857), this a problem with the soft material on the female side of the densely packed SCSI connector. If one male pin gets bent, it scars the female connector in a way that just makes more pins bent upon re-insertion, spreading the flaw to other channels. Fil will open a work permit to (a) reseat and bolt down the I/O chassis backplanes, and (b) mayhaps replace the connectors if need be I'll update the FRS ticket.
Sheila Kiwamu Patrick
We looked at a couple of the locklosses that people had from NOISE_TUNINGS last night. We found that the problem is when the ISS second loop becomes DC coupled it saturated. The attached screen shot shows some of the symptoms, where MC2 trans power is ramping up right before the lockloss, the IMC guardian changes state to turn off the ISS, the AOM driver mon and ISS second loop outputs have dramatic changes in the seconds before the lockloss.
I followed the instructions by Jeff K and D Sigg in alog 31942 to reset the ISS diffracted power. (H1:PSL-ISS_REFSIGNAL and H1:PSL-ISS_SECONDLOOP_AC_COUPLING_OFFSET)
I am not sure if turning on the ISS 3rd loop (done to help avoid CSOFT Pitch problems over the weeked) contributed to this being a problem last night.
In this afternoon, after a lockloss at NOISE_TUNINGS, we updated the offset (H1:PSL-ISS_SECONDLOOP_AC_COUPLING_OFFSET) again following Kissel's instruction. We set it to -0.12 which had been +0.05. However, this tuning wasn't good enough -- it still caused a power variation as the DC coupling was engaged by 10% or so. In the end, we set it to -0.095 as a result of a few iterative trials.
Note that when the offset was at +0.05, this caused a reduction in the power sent to the interferometer as the DC coupling was engaged. When it was -0.12, it increased the interferometer power instead.
2nd loop AC coupling offset H1:PSL-ISS_SECONDLOOP_AC_COUPLING_OFFSET is fine as of now.
No need to adjust. The fact that the diffraction stays at about 4% when the 2nd loop is ON and AC coupling is ON means that the offset is good. (That offset should not have any impact on the kick to the ISS when you turn off AC coupling.)
The problem seems to be the timing you turn on the third loop.
When the third loop is on while the 2nd loop is AC coupled, AC component of the AC coupling output becomes huge. We're talking about 10-something% RMS in the equivalent RIN (see the first attachment, top left, beginning of the plot).
In that plot, when AC coupling is turned off, instead of settling on the average of 6000, it settled on 6400. Suddenly the DC increased by a factor of 1.07, and this is seen by the second loop PDs (Ch15).
The output is held at the exact time when the AC coupling is turned off (except that it still slowly changes due to power scaling). Since the AC component is so huge, each time AC coupling is turned off, there's a big chance that the output is held at some number that is totally off from the average.
Smooth DC coupling transition requires that the AC coupling output settles on the value close enough to the average. This is only possible when the AC component is small enough to start with. Enable 3rd loop after ISS is DC coupled. If that's impossible we need a fancier solution.
If you look at the 2nd attachment, which looks at the same channels for 7 days, you can see that the AC coupling output never got huge until the 3rd loop was put in place in the guardian. It seems like we got 2 failures out of 3 trials.
Guardian changed, no 3rd loop for now.
With a telephone support from Sheila, ISC_LOCK.py line 3184 was commented out so that the third loop is NOT turned on during INCREASE_POWER.
We aren't turning on the 3rd loop in the guardian. If you think that CSOFT instability comes back, for now follow the instruction in alog 37046 and manually turn on the 3rd loop. You'll risk kicking yourself out, but if the noise is terrible you might choose to gamble.
I forgot earlier on the phone, we commented out the line in the guardian which requests the third loop, but this isn't enough to make sure it doesn't come on. We also have to turn off the input. The reason is that in Noise Tunnig the request for IMC_LOCK is set to ISS_DC_COUPLED, which will require it to pass through ISS_TR_CLOSED, which turns on the third loop unless the input is switched off.
