Guardian changes summarized:
Other changes:
The 16-hour lock from Jan 3rd hovered around 78 Mpc BNS range but had very strong scattering as described in 46222. This is a response to Jenne's request in that alog for detchar help on where the scattering might come from. This scattering is very broad and reaches to frequencies as high as 80Hz in strain, while it is seen at mostly 10-20Hz in auxiliary channels. The most strongly related channels to this scattering are Y-arm related, and the motion of ETMY's upper stage matches the fringes qualitatively.
Fig 1 shows a normalized spectrogram of (cleaned) strain with about 10 scattering events in 2 minutes. Hveto identified a strong correlation with the channel ASC-Y_TR_B_PIT_OUT_DQ. Glitches at 10-20Hz in that ASC channel are coincident with glitches 40-70Hz in strain. Note that this channel only sees the very loud/high frequency scattering.
Fig 2 shows the characteristic SNR vs frequency behavior of the scatter, where scatter that reaches higher frequency has higher SNR (as the detector noise drops to higher frequency).
Fig 3 shows a montage of three omega scans of scattering events, a large one (the same as the first one in fig 1), a medium one, and a weak one. The fringes are very broad in h(t), and reach ~80Hz in strain. The fringes are at lower frequency, narrower and stronger (higher SNR) in ASC-Y_TR. There are also fringes in ETMY Oplev Pitch channels with similar period at even lower frequency, but as you can see, these don't really line up.
Figs 4,5 show spectrograms of ASC-Y_TR_B_PIT_OUT_DQ with fringe frequencies calculated from the long motion of the upper mass* of ETMY and SRM suspensions. Out of all H1 suspensions, ETMY M0/R0 and SRM line up best. Fig 5 is an alternative look at a different time from Guillermo.
Finally, if I understood Jenne's alog correctly, they were intentionally moving the compensation plate from roughly 17:30-19:30 UTC, so I ignored those times when doing this work.
* As Rana pointed out, there are no laser beams on the upper mass. We're using these for convenience because the long signals there are calibrated to um, but we'd love to switch to an estimated lower mass displacement w/r/t something (to calculate fringe frequency) and are very open for suggestions.
TITLE: 01/04 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
INCOMING OPERATOR: None
SHIFT SUMMARY: Struggled most of the day to get to/past DRMI due to high microseism and wind and ALS issues. Just getting close to NLN at the end of the shift.
LOG:
16:28 Chris clearing tumbleweeds along X arm
16:35 Ed to EY to tweak OpLev laser
17:10 Ed back
18:10 Kyle to LVEA retreiving pump carts
18:25 Kyle out
19:23 Kyle to both mid stations
20:40 Kyle back
20:44 Fil to CER mezzanine
20:55 Fil out
This is a quick companion to my post for LLO 42583. Please look through or see the associated https://dcc.ligo.org/G1900011 that for context on these plots
I ran the same pipeline for H1 at 2019-01-03T08:00:00 UTC averaged for 700 seconds. I also looked at O2 data, but the machine did not appear to be sufficiently shotnoise limited around the UGF to compare with LLOs O2 in the presentation. Now it is.
darm_loop20190103T080000.png : the DARM loop I get from the RCG calibration system. I do not know yet why it is a poor estimator at low frequencies given A: the coherence, B: that the same thing works fine at LLO. May be from more cal lines in LHO, but those shouldn't have a broadband effect. In any case it works OK above 70Hz. I'll dig in to figure out why this is.
strain20190103T080000.png : The strain averaged over this time, from both cal systems.
DCPDCL20190103T080000.png : DCPDs in closed loop
DCPDOL20190103T080000.png : DCPDs in open loop, zoomed way out
spec20190103T080000.png : The PSD of the open loop DCPDs, calibrated to shot noise. as well as the noise-subtracted version.
I might say that the same loop-injected excess I'm curious about at LLO is also occurring at LHO. Curiously, the same .95 correction factor to the ASD as I used for the absolute calibration at LLO is also necessary for LHO, suggesting that the power/transimpedance calibration is 10% off.
loop_adjusted.png : If I try to virtually retune the loop correction, I do not get a flattening of the noise curve the way I can in the LLO O2 data shown in the presentation.
Once we see a sufficient segment of squeezing (or antisqueezing), I will analyze further.
Keita, Sheila, Georgia, Jenne, Craig
We noticed that with the ground motion a little high, our X arm green transmissions sometimes dip low enough to cause ALS to have trouble locking, because the drive to the UIM was causing the test mass to yaw. The first attachment shows a time series, the dips in the transmitted green power seem to correlate well with excursions of the optical lever in yaw, which are very well correlated with the length drive to the UIM.
We looked back a little at the history of this stage's coil balancing and decoupling since the vent: Two of the magnets were found to be flipped in June 42723 The coils were balanced in July 42740, measurements were made for the frequency dependent decoupling at the end of July 43150 and at that time Hang did the fitting for the pitch measurements and installed a L2P filter, but the yaw measurements weren't fit.
Based on the first attachment, the L2P decoupling is working well. We noticed while doing driven measurements that the pitch mode of the suspension rings with any transient for a long time, but things seem to be OK when we are just looking at the ambient ground motion.
Since we are concerned with motions that are at 40 mHz or so today, we tuned the DC gain of the L2Y filter without engaging any frequency dependent filter. We reduced the coupling seen by the optical lever by a factor of 4 or more by setting the DC gain to -0.00075
SQZ locking is now automated from locking OPO to SQZ angle. The automation will also bring things back up to nominal states when they're kicked out of place. SQZ guardians now have a manager called SQZ_LOCK (Haocun wrote this a while back, now it's ready). SQZ_LOCK manages OPO guardian, PLL (beat note locking), CLF, and LO. SHG guardian is still a stand-alone guardian (I see no advantage of it being managed at the moment).
When the nominal state is requested, SQZ_LOCK will lock OPO on CLF double resonance. Once the node is arrived SQZ_LOCK takes PLL to CHECK_BEATNOTE_2 which put SQZ pump laser frequency close to the IFO carrier and lock CLF. Although SQZ_LOCK manages LO guardian, LO lock still has to be requested by hand at the moment (note that LO_LOCKED_OMC doesn't work yet, so don't go there, the current nominal state is LO_LOCKED_HD).
LO looks at average RF3 beatnote value to determine if it's okay to close the loop. Once it's locked it uses stddev to determined whether or not it's actually locked. LO guardian also looks for rails. If any output rails it will disengage the loop and turn off boosts. In addition it also looks at OPO, CLF, and IMC status.
PLL looks at both OPO and IMC guardian states. If either is not nominal it will go to a state where it waits for both of them to be nominal again. IMC when loses lock usually kicks the reference cavity which is where our PSL LO sample comes from. When PLL is disengage LO will also rail and disengage. This has been tested with several IMC lock losses. All of them successfully came back to nominal state on their own.
Intensity stabilization servo is now a function lives in SQZ_OPO guardian.
The repeated cause of TTFSS EOM railing is LO output rails. So OPO CHECK_EOM state will now close EOM loop if LO outputs are okay and automatically return to nominal state. That will also allow PLL and LO to go back to their nominal states.
Will need some time with the IFO to get the LO OMC guardian to work. And will continue to fine tune the guardian.
We had difficulty locking most of the day today, but we were able to get to low noise with the 30W input power, which seems stable.
Difficulty locking
This morning we attempted to inject squeezing into the interferometer, which caused a lockloss. We've had a tough time re-locking all day since then, it seems that the arms are moving a lot at 50mHz (up to 5um pp). While it has been windy today it doesn't seem like the ground motion and tilt are large enough to explain the difficulty we've had. We have had the seismic in the "WINDY", we have had max wind speeds around 30mph for most of the day but periods when it was gusting up to 40 mph. There have also been a few rather small earthquakes.
ALS changes reverted
We spent some time having difficulty engaging ALS DIFF, we would loose lock reliably when it first ramped on the gain. We have reverted a series of changes that were made recently, and things seem better now. Yesterday we added a 120Hz notch to the 60Hz notch we use with ALS because we had bad ground loop problems at the PSL, today I removed this (46213). A couple of months ago people made ALS DIFF and COMM lock simultaneously to save time, this doesn't save us much time and make it confusing to debug things, so we have gone back to locking DIFF then locking COMM. Lastly, we went back to ramping the DIFF gain on first to 1/10th of its full bandwidth slowly, then ramping on to the full gain (this was removed over the break 46146). We also reduced teh gain back to 400 (rather than 600, the 3dB increase Rana mentioned in his alog).
Squeezer automation
We took some time to work on the automation of the squeezer while we were having trouble locking. Nutsinee spent some time working on the squeezer guardians, and has a working guardian to close the LO loop which she tested on the homodyne. We also spent some time testing some of the code that Haocun wrote for doing initial alignment of the squeezer, which is in the ALIGN_IFO guardian. We don't know if we will really need to do this each time we do initial alignment, but since we have been doing it each time we inject squeezing it will be useful to have it automated. We also added another branch to the ALIGN_IFO guardian that lets us engage the AS centering loops with a single bounce PSL beam, and offload the OM sliders, so that we know we are aligning the squeezer to the right alignment when we inject it.
ISS problems
Keita and Jenne spent some time looking at problems with the ISS, I don't know all the details but when the second loop is either off or AC coupled the first loop diffracted power is fluctuating a lot. Keita and I tried DC coupling the ISS with the interferometer at 30W input power, and DC coupling it caused the power to drop by about a Watt. The first attachment shows that the problem is when the AC coupling output gets held it is not at the mean of the perviously noisy value, which we think is why the diffracted power changed. We also saw that the diffracted power servo wasn't really doing anything, Keita increased the gain from 0.05 to 5. This was tuned for 2W, so it makes sense that we need to adjust it for 30W input power. This gain is now adjusted in the IMC_LOCK guardian before it turns on the ISS at 2W and before it gets DC coupled. The ISS second loop is left commented out of the guardian, because DC coupling it could still cause a transient in the input power while the first loop is so noisy.
30Watts input (~90kW circulating power) low noise
Following the instructions given in 46190 we were able to get to 30W without an issues, and then go to low noise. I moved ADJUST_POWER to after CHARD blends and it will not set the power to 25 W in one step, wait 5 minutes then set the power to 30W and wait 10 minutes. I didn't try to shorten this time, and I haven't tested the code this way, but think it should work. I also lowered the amplitudes of the HARD yaw dithers another factor of 10 since those lines are very large in DARM once we are in low noise.
We were locked in low noise (the only thing I skipped was reducing the 9 MHz modulation depth) from about 5:27 UTC to about 5:48 UTC. We had a lot of non stationary noise at low frequencies, similar to last night. I tried to drive ITMX CP to see if I could find a better alignment for it, but that unlocked the IFO. DARM has coherence with MICH below 100 Hz (attached plot), we haven't re-tuned the feed-forwards yet for this these settings.
TITLE: 01/03 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
INCOMING OPERATOR: None
SHIFT SUMMARY:
LOG:
17:29 Kyle to MX
17:59 Kyle to MY then back to MX
18:31 Kyle back
21:20 Ed to EY tweaking OpLev laser
21:49 Ed back
21:49 Betsy inspected TCS tables in LVEA for the suspected water leak. No water was found on or around the tables.
The current-monitor voltage used as a metric to adjust power was adjusted from 924mv to 914mv in a second attempt to quiet the glitching. OpLev damping is not being thus far on the ITMs and ETMs, however it is being considered for use in the not-so-distant future.
in other news...there is promise for the newest addition to the laser family being installed in the ITMX position by next Tuesday as it is passing stabilization testing in the LSB optics lab. Currently, the ITMX OpLev is a glitchy mess without a temperature stabilization enclosure (modified ice chest), the new standard of supply power, or armored fiber.
I checked spectrograms this morning after the power adjustment i made yesterday. Since the swap on 12/4/2018 there was a large improvement which was made better by the first adjustment on the 18th. Since then I made two adjustments. It seems that the glitchiness isn't any better than it was prior to the adjustment made on the 18th. I returned the power level back to what it was on the 18th and deduce that it "is what it is" until it can be replaced. ITMX is 1st priority and as it stands, ETMY is 2nd.
At 13:11 PST h1nds1 died. The console output was showing a kernel problem in the network stack. The only recourse was to reboot the computer using the front panel reset button.
This is a 2.6.35 kernel machine, but had only been running for 192 days so this does not look like a 208.5 day bug crash.
h1nds1 is the default nds server for the control room, so some nds clients may need to reconnect to the NDS server.
Looking at the other 2.6.35 kernel DAQ machines' uptimes shows:
| h1dc0 | 192days (16 days to go) |
| h1tw1 | 211 days (+3 days over!) |
| h1broadcast0 | 199days (9 days to go) |
I'm opening a WP to cover rebooting this machines next tuesday. h1tw1 may crash before then.
At 14:08 PST the daqd process on h1nds1 died with restransmission type errors. Logs do not show excessive data requests at the time. Monit was not monitoring the process for unknown reasons, so it did not get restarted.
Jonathan and myself got monit to start the process and it appears to be ok now.
And lastly, trended data was not available after the DAQD restart, restarting NDS resolved this.
While we were locked (16.5 hours!), we noticed that especially with the large wind (and / or anthropogenic ground motion?) we were seeing big scatter shelves in the DARM spectrum.
As a first pretty passive check, we moved the ITM compensation plates to see if a different position would give different scattering structure. The test positions (slider values shown in time series attachment) are perhaps slightly worse than the nominal, but it's pretty hard to say.
In the attached 5000 second spectrogram, I have drawn colored lines along the top of the x-axis to indicate the state of the CPs at various times. The plot includes 1000 seconds at the CPs' nominal positions (yellow line), a test position (turquoise line), again the original positions (yellow) and again the test positions (turquoise). During the times of the black lines the CPs are being moved. You can see that the first time we were in the test positions things were clearly worse, but that could perhaps have corresponded to some higher wind speeds (we've been getting sustained winds of more than 20 mph this morning). When we repeated the test back to the test positions, the scattering is maybe a teeny bit worse than the nominal positions, but it's really very marginal.
We moved on to some squeezing tests after this, so we haven't tried any more CP positions. For now, we're leaving them at their nominal positions.
If DetChar has time, it would be helpful to run the scattering identification tools on some of the data in the last ~5 hours, to help us identify what might be the cause of this scattering.
Looks good. Pitch & Yaw are within acceptable range (i.e. no centering needed).
Note: Script would not run from the OPS/WEEKLIES medm. Get message: "bash: /opt/rtcds/userapps/trunk/sys/h1/scripts/oplev_trends.py: Permission denied". So ran script manually.
Possible Issues: NONE to note
Laser Status:
Front End Power is 32.18W (should be around 30 W)
70W Output Power is 71.57W
Front End Watch is GREEN
70W Watch is GREEN
PMC:
It has been locked 0 days, 18 hr 8 minutes (should be days/weeks)
Reflected power = 10.2Watts
Transmitted power = 55.36Watts
PowerSum = 65.57Watts.
FSS:
It has been locked for 0 days 0 hr and 11 min (should be days/weeks)
TPD[V] = 2.529V (min 0.9V)
ISS:
The diffracted power is around 1.7%
Last saturation event was 0 days 0 hours and 12 minutes ago (should be days/weeks)
Figures attached. (Oddities of note are in bold below.)
For BSCs:
The mount for the PZT in the IO path on the PSL, IO_MB_M4, was swapped from the 2" mirror and the damped mount to the new PZT mount.
Aligning with the new PZT mount is challenged by having to remove the entire mount from the table, to access the bolts underneath, to adjust the +/-X position. There was also a change of the beam alignment when securing the bottm plate to the table. The beam was carefully aligned to IO GigE camera 2 and camera 3, and when the bottom plate of the new mount was secured to the table, the beam was gone from both cameras, and an offset was introduced into the IMC_IN beam. With the PZT it was possible to restore a beam to IO GigE 2, the camera that is on the bottom periscope transmitted beam. It was not possible to restore both camera images with the PZT. This indicates that the change in IMC_IN position and angle are due to the PZT swap. It was past noon, so the decision was to leave this change as is, lock the IMC, and evaluate how it and the beam downstream was affected, instead of using M3 and the PZT on the PSL, to restore an image to both cameras. The change in IMC_IN on the iris at the bottom periscope was identifiable as yaw with an IR viewer.
The reference beam from the shutter (between the PSL and HAM1) was recorded before and after the PZT swap. Those pictures show the difference is between the before and after beams is 1mm and 2mm in pitch and yaw (image attached).
Sheila and JeffK moved IMC DOF, Corey and JeffK started an initial alignment, and now the effort is to lock.
BEFORE reference time 12/11 18:00 UTC, AFTER reference time 12/12 1:24 UTC. A snapshot showing IMC WFS, IM4 Trans, and ISS Second Loop QPD signals is attached.
- Cheryl, Jason, Keita
Two drawings. The first shows the layout of the IO GigE cameras 2 and 3, both on the bottom periscope transmitted beam (IMC_IN). The second shows the change (exaggerated for clarity) of the front face of the PZT mirror (placed on the table vs secured).
This mount swap is in reference to IIET Ticket 5132.
The new mount is documented by the below details: