TITLE: 06/28 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 66Mpc
INCOMING OPERATOR: Cheryl
SHIFT SUMMARY:
Quiet shift. H1 locked for 11+hrs @ 68Mpc
LOG:
It seems that Seismon hasn't updated since the EQ near Indonesia yesterday
Sorry, that was me. I was trying to update our code to use code from LLO. Looks like I still need some help from Keith to get it working.
Which I have now updated to _V2 - need to check in code modules to SVN
H1 locked and observing for 7hr20min @ 68Mpc. There were a couple of glitches and no sign of ASC pitch instability. Environment is calm and DARM looks good.
Everything looks normal. Evidence of current adjustments made to the HPO Diodes last week are the only thing of notice.
TITLE: 06/28 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 68Mpc
OUTGOING OPERATOR: Travis
CURRENT ENVIRONMENT:
Wind: 16mph Gusts, 12mph 5min avg
Primary useism: 0.04 μm/s
Secondary useism: 0.09 μm/s
QUICK SUMMARY:
TITLE: 06/28 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 68Mpc
INCOMING OPERATOR: Ed
SHIFT SUMMARY: Some trouble getting to NLN during the first half of the shift until Keita disabled the ISS 3rd loop. Since then, we have been in Observing for 3 hours. No other issues.
LOG: See previous aLogs.
I remembered that there is one step to disabling the 3rd loop that isn't in the guardian. We need to turn off the input to the third loop filter bank by hand before Noise Tunnings, so that it will not be turned on by the guardian as it is written now. I am not sure if Keita already did this and accepted it in SDF, but I forgot to mention it on the phone.
The annulus pressure in HAM 11,12 (diagonal volume) is rising and causing the both AIPs to fall out of range. Both AIPs were replaced in May 2016. Today I leak checked all door and flange outer o-rings on HAM 11, 12, and also AIP conflat flanges. Found e-5 Torr-L/s of He range leak at bottom of HAM 11 south door and e-7 Torr-L/s of He range leak on HAM 12 south door. Kyle tightened bolts but flanges were already metal on metal. Suspect there is an imperfection on outer o-ring seal. Will address next time we have the opportunity to vent (or decommission) the diagonal volume.
FRS 8411
Finally back to NLN and Observing.
Lockloss 2:43 UTC. Cause not obvious.
Back to Observing at 3:55 UTC.
After retuning the SRCL FF yesterday (37150), Travis eventually got the interferometer locked after some problems with a windstorm, the CARM offset reduction, and eventually the ISS second loop. That lock in the first hour or so of Ed's shift had a better range than we have had in quite a while.
The time for the better spectrum below is June 27th 2017 7:39:16 UTC, while the "before" time is June 26th 2017 19:25:21 UTC. It would be interesting to take a look at a long bruco report for the after time. The jitter is a little lower in the after time, which is probably just due to an alignment drift. The range at this time is the best we have had in a long time, as shown by the 90 day trend in the second screenshot.
Ran A2L while LLO was still down. After A2L completed, we saw a lot of noise in the DARM spectrum below 100 Hz. After a few minutes of the commissioners investigating this noise, we lost lock. Running IA now in hopes of a better lock next time.
Jeff, Dave:
This morning during maintenance Jeff successfully ran the charge measurement script on ETMX and ETMY. I'll let him fill in the details, but this closes the LHO part of FRS8218. We suspect the core problem was an nds2 which has since been resolved.
The "red herring" SIStrLog error turned out to be a true error, fixed by opening up file permissions in the /ligo/logs/h1/inject_log area. I've opened FRS8405 to cover this.
I'm not sure this is common knowledge, but every time an awgstream runs it logs in a daily log file, for example
/ligo/logs/h1/injection_log/2017/06/27/injection.txt
...
1182627394.000000 H1:SUS-ETMY_L3_ESDOUTF_LL_EXC start zotws10 16523 1 awgstream ./test_sinewave_ETMY.txt 1
1182627418.059011 H1:SUS-ETMY_L3_ESDOUTF_LL_EXC stop zotws10 16523 1
1182627430.000000 H1:SUS-ETMY_L3_ESDOUTF_UR_EXC start zotws10 16536 1 awgstream ./test_sinewave_ETMY.txt 1
1182627454.055341 H1:SUS-ETMY_L3_ESDOUTF_UR_EXC stop zotws10 16536 1
1182627466.000000 H1:SUS-ETMY_L3_ESDOUTF_LR_EXC start zotws10 16548 1 awgstream ./test_sinewave_ETMY.txt 1
1182627490.054266 H1:SUS-ETMY_L3_ESDOUTF_LR_EXC stop zotws10 16548 1
Looked at the ~-4300 ct offset on the ETMY LL Fast I Mon channel. Cable was reseated at both the IO and AA chassis.
Offset dropped from ~-4300 cts to ~-250 cts. Noticed ADC interface cards on the IO chassis are missing the mounting hardware.
J. Kissel, F. Clara Talking to Fil after he got back from this week, he has indeed fixed the FAST IMON channels that were showing the ~4300 [ct] from one of the ADC legs of the given channels being shorted, but it looks like upon reseating the cable, the connector then shorted some of the NOISE MON legs. As the original documentation points out (see LLO aLOG 1857), this a problem with the soft material on the female side of the densely packed SCSI connector. If one male pin gets bent, it scars the female connector in a way that just makes more pins bent upon re-insertion, spreading the flaw to other channels. Fil will open a work permit to (a) reseat and bolt down the I/O chassis backplanes, and (b) mayhaps replace the connectors if need be I'll update the FRS ticket.
Sheila Kiwamu Patrick
We looked at a couple of the locklosses that people had from NOISE_TUNINGS last night. We found that the problem is when the ISS second loop becomes DC coupled it saturated. The attached screen shot shows some of the symptoms, where MC2 trans power is ramping up right before the lockloss, the IMC guardian changes state to turn off the ISS, the AOM driver mon and ISS second loop outputs have dramatic changes in the seconds before the lockloss.
I followed the instructions by Jeff K and D Sigg in alog 31942 to reset the ISS diffracted power. (H1:PSL-ISS_REFSIGNAL and H1:PSL-ISS_SECONDLOOP_AC_COUPLING_OFFSET)
I am not sure if turning on the ISS 3rd loop (done to help avoid CSOFT Pitch problems over the weeked) contributed to this being a problem last night.
In this afternoon, after a lockloss at NOISE_TUNINGS, we updated the offset (H1:PSL-ISS_SECONDLOOP_AC_COUPLING_OFFSET) again following Kissel's instruction. We set it to -0.12 which had been +0.05. However, this tuning wasn't good enough -- it still caused a power variation as the DC coupling was engaged by 10% or so. In the end, we set it to -0.095 as a result of a few iterative trials.
Note that when the offset was at +0.05, this caused a reduction in the power sent to the interferometer as the DC coupling was engaged. When it was -0.12, it increased the interferometer power instead.
2nd loop AC coupling offset H1:PSL-ISS_SECONDLOOP_AC_COUPLING_OFFSET is fine as of now.
No need to adjust. The fact that the diffraction stays at about 4% when the 2nd loop is ON and AC coupling is ON means that the offset is good. (That offset should not have any impact on the kick to the ISS when you turn off AC coupling.)
The problem seems to be the timing you turn on the third loop.
When the third loop is on while the 2nd loop is AC coupled, AC component of the AC coupling output becomes huge. We're talking about 10-something% RMS in the equivalent RIN (see the first attachment, top left, beginning of the plot).
In that plot, when AC coupling is turned off, instead of settling on the average of 6000, it settled on 6400. Suddenly the DC increased by a factor of 1.07, and this is seen by the second loop PDs (Ch15).
The output is held at the exact time when the AC coupling is turned off (except that it still slowly changes due to power scaling). Since the AC component is so huge, each time AC coupling is turned off, there's a big chance that the output is held at some number that is totally off from the average.
Smooth DC coupling transition requires that the AC coupling output settles on the value close enough to the average. This is only possible when the AC component is small enough to start with. Enable 3rd loop after ISS is DC coupled. If that's impossible we need a fancier solution.
If you look at the 2nd attachment, which looks at the same channels for 7 days, you can see that the AC coupling output never got huge until the 3rd loop was put in place in the guardian. It seems like we got 2 failures out of 3 trials.
Guardian changed, no 3rd loop for now.
With a telephone support from Sheila, ISC_LOCK.py line 3184 was commented out so that the third loop is NOT turned on during INCREASE_POWER.
We aren't turning on the 3rd loop in the guardian. If you think that CSOFT instability comes back, for now follow the instruction in alog 37046 and manually turn on the 3rd loop. You'll risk kicking yourself out, but if the noise is terrible you might choose to gamble.
I forgot earlier on the phone, we commented out the line in the guardian which requests the third loop, but this isn't enough to make sure it doesn't come on. We also have to turn off the input. The reason is that in Noise Tunnig the request for IMC_LOCK is set to ISS_DC_COUPLED, which will require it to pass through ISS_TR_CLOSED, which turns on the third loop unless the input is switched off.
