Especially with the DRMI needing alignment help during most re-locks, I have created a new state to help check BS-only alignment.
Recall that if the DRMI flashes don't look so good, you can select the PRMI_Locked state that Sheila created some time ago. This state will misalign the SRM and attempt to lock the power recycling cavity with the arms held off resonance.
Sometimes though even these flashes aren't very good, and the PRMI isn't catching lock. Now you can select the new state Check_MICH_Fringes in the ISC_LOCK guardian, and it will misalign the PRM. This state is currently not set to try to lock MICH, so it'll never look quite like MICH_dark_locked from initial alignment. But, you can watch the flashes on the AS air camera, and move the BS until they are as circularly symmetric as possible. Note that sometimes you'll only see a few flashes per minute, so this requires a bit of patience, but it's often still faster than giving up and doing a full initial alignment. Once the flashes on the AS air camera are circularly symmetric, re-select PRMI_locked. This will realign the PRM, and go back to the usual locking sequence.
The PSL tripped around 21:37 UTC, it took us about an hour to recover from this . We called Jason and Peter, Jason called back and walked me through restarting it remotely. I added about 200mL of water to the chiller, there was water on the floor in front of the chiller. Jason confirmed that this is normal, and that the turbulence and bubbles in the chiller tube are from a known slow leak.
Before restarting all the models on OAF, I accepted diffs on CAL CS, OAF, and SUS proc PI. a screenshot is attached for PI.
There were two periods of unexplained motion of ITMY during undisturbed overnight lock on Sep 23. Both ISIs, plus all levels of the suspension (including oplevs) saw it, but there's nothing evident in the HEPI sensors or the ground. These events made two periods of large noise in DARM (first plot), a few minutes apart, and a few minutes before lockloss due to PI. At that point, the PI didn't seem to be having any effect. The full Omega scan shows that ITMY has lots of anomalous motion. The second-stage ISI GS13s see a quick transient with rich structure (plots 2 and 3). The ST1 sensors see something similar but much less distinctly. The L1 OSEM clearly sees a line below 1 Hz rung up for more than a minute afterward (plot 4). I was not able to find anything on the ground sensors or HEPI that could have rung this up. I haven't looked as closely at the second event. I think this warrants more followup by someone familiar with the seismic isolation.
Kiwamu, Terra, Lisa 3 earthquakes, 2 user errors and 1 PI gave us the opportunity to extensively test our lock acquisition sequence today. (It works pretty well actually. The weakest part is the DRMI lock: the interferometer ends up always misaligned right after an unlock, and it requires manual adjustments all the times to get DRMI to lock. Once the DRMI locks, than the locking sequence rarely fails.) Today we explored the (allegedly) jitter/intensity noise as function of interferometer alignment (SRM alignment, CSOFT/DSOFT and PRC1 offsets). More details will follow, but the bottom line is that we have been able to increase the recycling gain (max 31, typically 28-29, with PRC1_Y offset= -0.06, CSOFT_P = +0.120), the cavity pole (360, typically 340), and minimize "jitter", but we have not been able to do everything at the same time. All the noise tuning steps (including the now standard CO2X 240 mW power) are in the guardian, except the IMC WFS offsets. We also run A2L. We leave the interferometer locked with the SRM aligned by hand to minimizes jitter noise (ASC still open), with recycling gain 29 and cavity pole ~340 (as measured by CAL_CS_TDEP_F_C channel). The range is approaching 70 Mpc. Observe intent bit set at 9:17 UTC, Sep 25.
P.S.: The summary pages are not working for H1 (they do for L1): https://ldas-jobs.ligo-wa.caltech.edu/~detchar/summary/day/20160925
Modes 17 (15542 ETMX) and 25 (15541 ETMY) continue to give us trouble; about three hours into the lock Mode 17 will get high enough that Mode 25's PLL will jump to it, after which Mode 17 rings up quickly. Matt had set up a lower bandwidth loop in the PLL of these to help prevent this and yesterday I tightened it up even more. We've still lost lock the past two nights from these modes. To this end, I'm leaving Mode 25 in BYPASS with a 1Hz BP (still wider than I've seen it shift in frequency over many locks).
Peter (on phone), Kiwamu
We restarted the PSL at around 3:20 local with no problem. Later, I added 130 ml to the crystal chiller which was found to be beeping for a low water level. This cleared an error message on the display and the beep sound.
Kiwamu, Nutsinee
We did a quick test with the HWS yesterday hoping it might fix the issue of HWSY sled reflected off the CP (alog29905). We first moved the ITMX and ITMY CP by a few hundred counts to confirm that only HWSY has sled reflection from CP. Then we swapped the X(790nm) sled and Y(840nm) sled then repeat the process. HWSY still had most of its reflection from the CP. The conclusion is, using different wavelength didn't matter.
We removed the HWS plates and stopped the code. This configuration still remained.
During this test we noticed that HWSX looks clipped. Not sure when did this started happening but Kiwamu said the data from HWSX he's been using make sense. No major misalignment to the SR3, ITMX, and BS in the past few days. A small touch to the top and bottom periscope fixed this clipping.
The sled has been swapped back. This happened on Sunday. HWS plates are still off.
Josh, Andy
Summary: The ASC AS_C INMONs were overflowing on 9/23 but not 9/24. We're not sure if this has any negative consequences.
During O1 we were asked to report any overflows that happened in analysis ready time. The summary page for overflows for 9/23 had a lot of entries but nearly all of them were at the lock loss. Of the handfull of things that overflowed for longer periods we looked at the ASC AS_C INMONs.
These overflowed the entire lock on 9/23 (like in fig 1) but were not overflowing in O1 (fig 2) and were not overflowing in the most recent lock on 9/24.
The histograms of the values for segs 2 and 4 are compressed around -32768 (though these are not integer counts and that value is exceeded) (fig 3).
We're not sure if this translates into problems for AS_C, but there is somewhat elevated noise in those channels now compared to O1 (fig 4,5) and those channels are used for ASC SRC2 which in turn has ~5-10Hz coherence with SRCL (fig 6).
But AS_C channels have not much coherence with DARM (fig 7).
We thought it was worth reporting even if there isn't an obviously bad consequence.
Title: 09/23/2016, Evening Shift 23:00 – 07:00 (16:00 - 00:00) All times in UTC (PT) State of H1: IFO is locked at DC_READOUT. Environmental conditions are not that favorable. Winds are a Fresh Breeze with gusts to Near Gale (19-24 base, gusts to mid-30s). Seismic activity is elevated but still below 0.5um/s, microseism is also elevated and getting worse due to a mag 6.3 EQ in the Philippines. Set ISI-Config set to EARTH_QUAKE_V2. Commissioning: The commissioning team is working feverishly ahead of projected arrival of the EQ. Outgoing Operator: TJ Activity Log: All Times in UTC (PT) 23:00 (16:00) Start of shift 23:10 (16:10) Kyle – Back from Mid-Y 23:41 (16:41) Lockloss – Mag 6.3 EQ near the Philippines 23:45 (16:45) Nutsinee – LVEA to adjust HWSX alignment 23:49 (16:49) Keita – Going into the LVEA 00:02 (17:02) Keita – Out of the LVEA 00:04 (17:04) Kiwamu – Tour in the LVEA 00:11 (17:11) Nutsinee – Out of the LVEA 00:30 (17:30) Kiwamu – Out of the LVEA 04:08 (21:08) Relocked at NOMINAL_LOW_NOISE with 51.0W & 53.2Mpc 04:30 (21:30) Set Intent Bit to Observation 04:52 (21:52) Lockloss – Mag 5.8 EQ near New Zealand 05:25 (22:25) ETMX – RMS WD tripped - Reset 05:26 (22:26) HAM6 – SUS & ISI WD tripped – Reset 06:25 (23:25) Locked at NOMINAL_LOW_NOISE 06:26 (23:06) Intent Bit Set to Observing 06:30 (23:30) Ending Shift Title: 09/23/2006, Evening Shift 23:00 – 07:00 (16:00 – 00:00) All times in UTC (PT) Support: Kiwamu, Sheila, Incoming Operator: N/A Shift Detail Summary: It was a struggle to relock the IFO after the Philippines earthquake. With much assistance and perseverance, were able to relock and eventually take the IFO to NOMINAL_LOW_NOISE. The environmental conditions are good. Wind is no more than a Light Breeze (max 7mph), Microseism is centered around 0.08um/s. However, seismic started to ring up due to a Mag5.2 EQ near New Zealand, which eventually broke lock. Was able to switch the ISI to Eaeth_Quake_V3 before Lockloss, but not soon enough to prevent the Lockloss. The New Zealand EQ was followed by back to back Mag4.9 & Mag4.7 earthquakes near the Reykjanes Ridge. Seismic recovered in about an hour. Much trouble relocking, and Roll Modes still a bit elevated. Finally, able to get to NOMINAL_LOW_NOISE with Intent Bit set to Observing. Will take a small success and call it a night before another earthquake comes along to break the lock.
Not much progress has been made toward understanding the ITM charge mystery of aLOG 29699 and earlier. I tried to get quad-specific charge information before the earthquake hit tonight, but I found the data had insufficient SNR at my excitation amplitude.
To measure effective bias voltage of an ITM, read /ligo/home/chris.whittle/Documents/ITM_CHARGE_README.txt, set appropriate excitation amplitude/frequency and run /ligo/svncommon/SusSVN/sus/trunk/QUAD/Common/Scripts/ESD_TM_charge.py as (for example)
python ESD_TM_charge.py -l -m -o ITMX
(further instructions in the readme).
To perform a swept sine measurement, set the bias voltage to your desired value, then open the DTT template at /ligo/home/chris.whittle/Documents/charge/itm_charge_sweep_sine. You will have to set an envelope for the excitation frequencies. Previously, /ligo/home/chris.whittle/Documents/charge/analyse_sweep.py was used to calibrate and re-plot this data (requires transfer functions and coherences from charge measurement and pcal measurement).
ISS has true in-loop signal called H1:PSL-ISS_SECONDLOOP_RIN_ERR1 that is picked off of the error point of the second loop board, and we also have pseudo in-loop and out-of-loop sensor called RIN_INNER and RIN_OUTER that are digital sum of indivisual PD output.
In the morning we noticed that pseudo in-loop and out-of-loop sensors have huge low frequency noise (first attachment, brown).
Three problems were identified:
The first problem will be solved by model change, but for now we disabled the scaling.
The second problem was solved by adding HPF in the RIN_INNER and RIN_OUTER.
After these, huge noise was gone, and now the RIN_INNER became smaller than RIN_OUTER, but the discrepancy between the RIN_INNER and the true in-loop signal didn't make sense (second attachment).
We went to the floor and disconnected the PD5-8 cable from the front panel of the chassis, and confirmed that all PD 5-8 signals went away, and got confident that INNER- and OUTER- are NOT swapped anywhere.
However, when I connected the cable back, PD5 signal had a huge offset, and the DC channel and the whitened channel were opposite in sign, which doesn't make sense. It seems like something happened on the transimpedance board.
We turned the power of the chassis off, disconnected the PD5-8 cable, and powered it on again, and the PD5 DC channel has a huge offset.
We did NOT put PD5-8 cable on PD1-4 connector as we feared to break PD1-4 channels.
The chassis will be pulled for investigation on Tuesday.
TITLE: 09/23 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Lock Aquisition
INCOMING OPERATOR: Jeff
SHIFT SUMMARY: Some morning work on the AS_C QPD whitening, then after that was done locking started but was severely slowed by some ASC issues when engaging DRMI ASC. Kiwamu narrowed it down to either SRC1 or SRC2. When he engaged these separately, it seemed to work, but when the guardian did it simultaneously, it would crash within ~30s.
LOG:
3pm local 12:15 to overfill CP3 with 1/2 turn open on LLCV bypass (exhaust bypass left open) *while I was out there, I closed bypass exhaust valve on CP4 from this morning's activity
Chart shows changes of the IM alignment over
IM3 yaw from O1 start to Current has changed by -111urad, highlighted in blue.
This morning, I have measured the in-loop and out-of-loop spectra of the second loop ISS when it was closed at 50 W with the recently-installed-boosts using an SR785. This was without the full interferometer but the IMC. Here is the results.
The purple, blue and red curves are all the in-loop spectra (i.e. SUM14) but with different frequency resolutions. The yellow, green and cyan curves are the ones for the out-of-loop sensor (i.e. SUM58). The interpretation and implication of this plot will be posted by Keita, Daniel and Ben later. The data, plotting script and figures are also attached as a zip file.
This is a follow-up study for 29524, 29556,
Synopsys- With the current CO2 and ring heater settings, the readout gain for laser frequency with REFL45I does not flip the sign any more during and after the power up.
Some details- Here is a screenshot showing the sensing gain for laser frequency at REFL_A_9I and REFL_A_45I from the last night. The sensing gain were monitored by the mod/demod technique at 900 Hz which is the same as what Stefan did in the past (29524, 29556).
Very similarly to the past observations, REFL_9I maintains a steady gain throughout the measurement. REFL_45I was also stable. Its sensing gain changed by a factor of two from the beginning of the 50 W operation to the end of the measurement (for about 35 minutes). As opposed to the previous observations, REFL_45I did not experience a large deviation of more than factor of two or sign flip. This is consistent with our experience in the sense that the interferometric loops can hold the interferometer locked without significant changes during and after the power-up.
Some notes on the thermal settings that we used in this measurement:
Jeff K, Sheila, Chris Whittle
Having recently taken measurements for the CARM and IMC loops with the newly installed 200 kHz pole (aLOG 29735), we have begun an investigation into whether we can boost the gain in CARM to help mitigate frequency noise in DARM without compromising loop stability. We used the following form of the CARM open loop gain:
G_CARM ~= g_CARM * H * ( g_IMC * G_IMC / (1 - g_IMC * G_IMC) )
where g_IMC and g_CARM are tweaked, and G_CARM and G_IMC were measured (see above). H is a combination of the electro-optical CARM plant of the IFO and the electro-optical IMC plant, and is calculated from the above measurements. We are ignoring the slow path here as we are far above the 30 Hz crossover. See aLOG 22188 for more details.
Note that the closed loop gain of the IMC was extrapolated back to 1 kHz from 10 kHz (assuming a unity CLG below 10 kHz).
The OLG of the IMC loop shows that we can't get away with an IMC gain much grater than 2 dB without hurting our phase margin too much. Similarly, the loop suppression of the IMC loop significant gain peaking above 2 dB.
With g_IMC = 2 dB, our CARM phase margin suffers above g_CARM = 2 dB. This gives us a a factor of 2.08 dB suppression in our CARM loop suppression. We therefore propose increasing CARM and IMC loop gains to g_CARM = 2 dB and g_IMC = 2 dB. Although this introduces some additional UGFs, all are stable, and the worst of which has a phase margin of 31.6∘.
The script for producing these plots can be found at:
Sheila, Chris Whittle
We made these gain adjustments, but couldn't see any changes in the DARM spectrum by eye. Nothing bad seemed to happen, however. We've left these changes in.
PSL tripped due to the power meter flow. The 1st attachment shows the power meter flow dropping just before the crystal chiller interlock trips. The 2nd attachment shows the power meter flow dropping ~1 second before the FE flow drops. Interestingly, it takes ~6 seconds after the power meter flow drops and the crystal chiller interlock trips for the crystal chiller flow to drop (3rd attachment). Not sure if this is due to the rate that the Beckhoff PC polls the crystal chiller, or if it really takes 6 seconds from interlock trip to chiller shutdown (I suspect the former). The full timeline of the trip, assuming t0 = 21:37:51 UTC (time of trip):
Filed FRS #6319.