TITLE: 07/10 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Earthquake
OUTGOING OPERATOR: Ed
CURRENT ENVIRONMENT:
Wind: 7mph Gusts, 4mph 5min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.06 μm/s
QUICK SUMMARY:
Laser remains off from trip. I see Jason in the PSL enclosure investigating.
TITLE: 07/10 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Earthquake
INCOMING OPERATOR: Patrick
SHIFT SUMMARY:
LOG:
15:57 Jeff B into Diode room to power cycle dust monitor. WP 7071.
15:58 begin damping exercise - ITMX mode 10 (996.252Hz)
16:14 Jeff back
16:30 Meeting in control room to discuss WPs
16:50 Richard, Jeff and Fil working in CER and Diode room with dust monitor WP7071
19:00 Vern + guest up to observation deck
19:45 Carlos up to observation deck
19:56 Carlos back
20:09 Corey to vacuum optics lab
20:55 SVN (SDF Diff) file commitments for previously modified files.
21:42 Lockloss - PSL trip due to flow error
22:01 Jason and Peter plan to go into the enclosure to investigate that PSL trip
22:48 Thomas and Marc to MY
J. Kissel, H. Radkins, J. Warner
We're continuing to attack violin modes post Montana EQ. I attach a spectrum of "our" (all Jim and Hugh) progress, where the reference (GREEN) is this morning. We can definitely see that the violin modes begin to get hefty non-linear shoulders by the time the RMS passes ~5e2 ADC counts, which is pretty surprising for a 32e4 count ADC...
New modes that Hugh and Jim have now successfully cooled today:
EX 1475.09 MODE7 YAW FM4(100dB), FM6(1475.09), G = -500
(This mode interacts with the "already solved" 1475.25 EX mode, so that damping must also be ON and some small level while damping this mode)
IY 1472.45 MODE8 PIT FM4(100dB), FM6(1462.45), FM7(-60d1.5k) G = +1000
IY 1470.38 MODE4 PIT FM4(100dB), FM6(1470.38), G = -500
IY 1470.82 MODE5 PIT FM4(100dB), FM6(1470.82), FM7(-60d1.5k) G = +1000
As before -- these are reported for the record, but the Violin Mode Table has been updated and that should be treated as cannon not this aLOG.
I've been continuing to manipulate the damping output of the problematic pair of modes at 1463.09 Hz, including
- Installing narrow bandpass in all test masses, exploring the unit circle with each of the 4 test masses, and all 3 degrees of freedom (L, P, Y)
- Went back to ITMX and focused energy there (since that;s what I was driving when I originally rung it up)
- Adding +/- 30 [deg] filters, and spinning around the unit circle there
- Broadening the band pass, in hopes of changing the relative drive phase between the two modes.
After three hours of failure, I was *just* about to think I saw hints of progress driving the wide-band band pass in YAW, with FM4(100dB) and FM5(1463.09W) and +1 gain, but then we lost lock because the PSL tripped. Guh!
While wasting time staring at StripTool for 3 hours, I grabbed a 0.5 [mHz] resolution spectra to confirm the violin mode table's claimed exact frequency of these modes. Unfortunately, DTT (and foton's) cursor can't resolve past 2 decimal places. But at least zooming in visually, I can confirm that the two modes are:
1463.0961 +/- 0.0005
1463.0982 +/- 0.0005
We were focused on the 1.5k modes so far today, and made no attempts at progress on the remaining high 1.0k pair of modes at 1008.45 / 1008.49 Hz.
After careful analysis of high resolution spectrums of the "pair" of modes at 1463.098 and 1463.101 Hz I think I understand why commissioners are having issues damping them. In conclusion, the two modes have swapped order after the misalignment caused by the Earthquake, If this has not been taken into consideration then the damping filters are being applied to the wrong mode.
This problem serves as justification of the importance of violin mode identification not only by mass but also by fibre.
Yesterday I commented on another alog saying that this "pair" while associated to the same mass (ITMX), they most probably were associated to different fibres (more concretely to FR and BL fibres), due to the fact that this pair was an order of magnitude closer than any other pair. A confirmation that this assumption is correct is shown next by looking at the pair of modes at 4 different times; (red) is during O1 Dec 2015, (blue) is a couple of days before the Earthquake during Observing mode, (green) is hours after the Earthquake and (purple) is about the time of the entry above.
Looking at red and blue plot we see how the change in frequency of both modes occurs in opposite direction which could not be possible if it was related to the same fibre, basically one is increasing in tension and the other one decreasing, probably due to change on suspension pitch confirming that one is a front fibre and the other one is a back fibre. Furthermore, notice the small change on frequency over 1.5 years, and the huge change caused by the Earthquake, presumably due to the considerable change in alignment caused by the Earthquake and consequent change in the fibres' tension. The Earthquake changed considerably the ground where the detector sits. But most importantly notice that the change in tension is so high that the position of the frequencies associated to the 2 modes switched such that now the lower frequency mode is actually the one that before was the higher frequency one and viceversa. If this has not been taken into consideration then the damping filters are being applied to the wrong mode, which may explain why it is getting hard to damp them.
Notice that the frequency change is once again decreasing as per the purple plot, presumably after ground relaxation after the Earthquake and mass pitch alignment going back to previous values.
Just to confirm this further I looked at spectrums (at the same times as above and same color coding) for the other two pairs of 3rd harmonic modes that have been identified as being associated to the same mass. We can see how for each pair the change in frequency takes place in same direction between them (so they are associated to the same fibre) and same ammount as above, but both pairs change frequency in opposite direction showing that these two fibres are again front and back.
First pair at modes 1456.18 and 1456.84 Hz:
Second pair at modes 1467.47 and 1467.96 Hz:
10:21 21:41UTC Jason and Peter are going into the enclosure for investigation.
Kiwamu, Daniel, (WP7066, ECR:E1700127-v1)
We modified the ASC front end model in order to accommodate lock-in oscillators which will be used for commissioning the new AS72 WFSs (37042). The conceptual design is summarized in T1700324. Additionally, we removed the parts that were for the AS90 WFSs because (A) we had not been using them and (B) we will take over the same electronics and ADCs for the AS72 WFSs. The below are the model files that were modified today:
They are checked into SVN. Note that Livingston is not using ASC_MASTER these days. The attached screenshots show the new components in the ASC model. As stated in the ECR, we are adding twelve new DQ channels as follows.
Also, we removed the following four DQ channels that needed to go out since we took out the AS90 system:
I was able to compile the model without any error messages. The model is ready to install tomorrow during the maintenance period.
The following files were commited to SVN:
There are files that are currently being modified by on site staff. They will respectfully be asked to commit their work.
There are some files that were modified by incoming operators. They will be asked to address their work as well.
Remnants of the last diode current adjustments are evident. The issues with the chiller plots is being dealt with. Everything else is ok.
Concur with Ed, except for the chiller plots everything looks normal. For more info on the chiller discrepancies see alog 37411.
FRS8255
Chandra, Dave:
The temporary high alarm leves for PT124 and PT144 was removed this morning (reduced from 5.0e-08 to the standard 5.0-e09 Torr) and the alarm system was restarted. In the process I discovered that the full alarm configurations for CP3 and CP4 were still commented out (we were using a subset). Now these cryo-pumps are fully functional, I have restored their full alarm configuration.
I've set FRS8255 into PENDING ready for close-out.
Richard, Jeff B, Dave:
the EPICS IOC which reads the diode room dust monitor was restarted, we are recording dust measurements again. The wiki page for the dustdr was found to have problems, which were corrected. Also the serial baud rate had been increased from 9600 to 19200, this change was made to the wiki page text. The startup script had an incorrect name, this was corrected.
J. Kissel, Operators
Where we left off on Friday:
- We were battling 2 (actually 4) modes,
ETMY: 1008.4502 & 1008.4938 Hz (Mode separation: 0.0436 Hz)
Unknown: 1463.0974 & 1463.1005 Hz (Mode separation: 0.00309 Hz)
For the the later unknown modes, I tried the following:
- Driving ITMX or ITMY, and even Both Simultaneously and for each of those configurations:
- Multiple bandwidth sizes
- Driving in Pitch, Length, and Yaw
- Gains from very little to about to saturate the DAC
- The usual phase rotation game
I attach a StripTool that shows the control signals of the damping filters all I was ever able to get was the beatnote between the two frequencies, with a period of 5.3 minutes. That extra long beat-note envelope makes it extra time-consuming to try new configurations, because one has to wait for one, if not several peaks of the envelope to assess whether a positive (or negative) change has been made.
- We also has somehow managed to ring up > 1.5 kHz harmonics, especially some 6 kHz mode
Where we pick-up today:
See attached spectra.
These same modes mentioned about continue to dominate the RMS, along with another set of modes (1475.2520 & 1475.0984 Hz) that has rung up again over the weekend, where the latter of the two (1475.0984 Hz) is also one for which we've not yet identified a filter and/or settings.
We're otherwise keeping the Violin Mode Table as up-to-date as possible with any success we have.
I have edited the Violin mode tables you linked to which it had already the mode 1463.097 identified as ITMX, I can confirm that the other pair mode 1463.1005 is also associated to the same mass actually to the same fibre. UPDATE: I have been asked to provide some evidence of this affirmation, so here it is. A couple of year ago (during O1) Evan Hall and myself were allowed to drive the PUMs of each test mass individually with the damping filters turned off for fundamental and higher harmonics, while the detector was locked (see original alog here). Next a reminder of times of injections:
Injection on ITMX: from 2015-10-28 20:39:44 to 2015-10-28 20:44:13, channel H1:SUS-ITMX_L2_DAMP_MODE5_GAIN
Injection on ITMY: from 2015-10-29 20:22:51 to 2015-10-29 20:31:17, channel H1:SUS-ITMY_L2_DAMP_MODE8_GAIN
Injection on ETMX: from 2015-10-29 20:37:20 to 2015-10-29 20:45:37, channel H1:SUS-ETMX_L2_DAMP_MODE5_GAIN
Injection on ETMY: from 2015-10-29 20:52:36 to 2015-10-29 20:59:11, channel H1:SUS-ETMY_L2_DAMP_MODE6_GAIN
Then I monitored change of amplitude and phase of the violin modes and harmonics before and after the injection (for each injection). I attach the plots for the harmonic lines at 1463.097 and 1463.1005 Hz. The injection starts at about second 1200 on these plots. It is unquestionable the huge difference observed when the injection was applied to ITMX respect to all the other cases.
UPDATE2: While I am 100% sure that 1463.097 and 1463.100 are associated to ITMX however my confidence of them being associated to the same fibre is not that high, I just made an obvious conclusion based on the fact that they are the closest pair of modes for the 3rd harmonic, by an order of magnitude!
They are 0.003Hz apart while the next adjacent pair of modes for the 3rd harmonic are 0.05Hz apart. Maybe actually the fact that they are much closer than any other pair shows the opposite conclusion, that they actually correspond to different fibres. In a way that one of the modes of each pair happens to be incredibly close.
And actually looking to the original identification of the fundamental mode to actual fibres (https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=17610) this may actually be the case. Look at the identification of modes 501.208 (associated to ITMX FR) and 501.254 (associated to ITMX BL), they are 0.046Hz apart, closer than any other pair of modes on ITMX. A difference in inharmonicity of the 2 fibres may explain why their 3rd harmonics are much closer than their fundamentals.
It may be worth trying to damp these modes by acting on those 2 fibres.
The other 2 modes you are having problems with; 1475.2520 it is unquestionable ETMX, and 1475.09 most probably is also associated to the same mass (probably a pair mode) but notice that this mode did not have a noticeable change in amplitude when driving any of the PUMs associated to QUADs, which means is weakly coupled explaining why you are having issues with damping filter settings.
At approximately 15:30 UTC (8:30 PDT) on Saturday, 7/8/2017, the flow through the HPO laser heads dropped by ~0.12 lpm; see the first attachment. This is worrying as the trip point for the laser head flow is set to 0.4 lpm; the flow through heads 2 and 3 is sitting at ~0.45 lpm, just 0.05 lpm above the trip point. Looking at the other channels in the cooling system the picture becomes a little clearer (maybe).
The 2nd attachment shows the pressures in the PSL cooling manifold; H1:PSL-OSC_PRESS1 is the pressure at the manifold input, H1:PSL-OSC_PRESS2 is the pressure at the manifold output. As seen in the attachment, the pressure at the inlet increased slightly, and the pressure at the outlet dropped slightly. The third attachment shows the flow through the HPO power meter circuit (H1:PSL-OSC_PWRMETERFLOW), the 35W FE laser circuit (H1:PSL-AMP_FLOW), and the overall flow out of the PSL crystal chiller (H1:PSL-OSC_XCHILFLOW); the flow out of the chiller dropped slightly (~0.1 lpm) while the flow through the HPO power meter and 35W FE circuits both increased very slightly (interestingly, the signal from the FE flow sensor cleaned up after the event, and the frequency of drops in the power meter flow sensor signal also decreased). The most likely cause of this behavior is some kind of flow restriction, probably somewhere in the laser head cooling circuit (although it is possible there could be a blockage in the filters under the PSL table).
The 4th attachment shows the temperature of the individual laser heads. All saw an increase of ~0.2 °C, which indicates that this is a real drop in the flow through the HPO laser heads. Fortunately this was not enough to effect the output power of the laser (5th attachment), and the temperatures have remained steady since the loss of flow on Saturday.
In the short term, we can try increasing the overall flow at the crystal chiller to see if it clears whatever is restricting the flow; at the very least we can increase the flow so we are a bit further from the trip point. We will also inspect the filters beneath the PSL table when we are in the enclosure during tomorrow's maintenance window. Any further investigation will require opening up the cooling system, which is highly invasive.
Filed FRS 8482.
LHO WP 7073.
I increased the flow out of the PSL crystal chiller from 21.4 lpm to 22.5 lpm. While this doesn't appear to have knocked anything loose, at the very least the lowest of the 4 laser head flows is now reading 0.5 lpm. This lessens the chance of a PSL trip due to low flow. I will leave the chiller at this flow rate until tomorrow, when we can take a closer look during the maintenance window.
J. Kissel Just posting an update on this -- the H1 PCAL Y RX PD reported displacement continues to decay, likely due to some temperature dependent clipping. This had been a part of the FRS Ticket 8328, but there's little we can do during an observation run to fix this. We've since closed the issue as LONGTERMFIX. AS such, I'll raise to an integration issue, and mark as WHEN VENT (similar to the promised work in Integration Issue 4700). For now, recall that we've switched over to using TXPD as the calibration reference (see LHO aLOG 37166).
Opened IIET Ticket 8481
I've attached my related notes on this subject, as I've been trying to draw up a history of the issue since the Fall. "Issue History.pdf" is the information I've been able to drag out of aLOGs and long time series, "Investigations.pdf" are the questions I've had and a select few I've answered. Apologies for the rambling nature, these are my personal notes. Short summary of my investigations: temperature correlation on the Rx side is quite clear but higher frequency coherences appear to be only in obvious places (other PCalY OFS and Tx channels, ETMY ISI channels) and I haven't figured out how to measure coherence for long term, slow frequency signals.
Further investigations and notes will now exist in DCC document G1701350 (see https://dcc.ligo.org/G1701350).
DQ Shifter: Alan Weinstein, Email: ajw@ligo.caltech.edu
LSC Fellow: Paul Marsh, Email: paul.mecheng@gmail.com
Full summary is here.
CW_GAIN issue resolved, see aLOG 37304.
CW GAIN issue has been resolved. The CW_GAIN and CW_TRAMP values were being monitored by Guardian, which forced the IFO out of observing when those values changed; those values were changed due to a loss of hardware injections and subsequent restart by the psinject script. Dave Barker and I have changed the gain and ramp to be unmonitored channels again. Notes attached.
Change undone after the decision was made that losing IFO lock was the desired behavior and issue was mitigated by sparse nature of psinject reset. See aLOG 37447.
