Meant to make a note of this earlier, but the Maintenance deluge took over.
H1 was locked this morning. Apparently it had been down, but went back to Nominal Low Noise (NLN) on its own!
While in NLN, received a Verbal Alarm at 8:14am (15:14utc) about the PI of Mode #25 was ringing up. I took the gain from +100 to -100, and this seemed to turn it around, BUT we did drop out of lock 5min later. Mentioned the alarm and addressing of the PI to Terra.
Installed and tested the scroll backing pump for the Vertex Turbo (part of a site-wide modification to main turbo pumps which allows selecting either QDP80 or scroll pump to back Turbo). Confirmed that the Turbo's fore-line isolation valve (aka "Safety" valve) closed at the loss of scroll pump control cable connection or AC at its motor winding, i.e. that the backing pump interlock functionality remained as originally designed. Also - Ran both Corner Station QDP80 pumps for ~2 hours and energized all four Turbo Pump controllers (done weekly to charge levitation batteries) Leaving Vertex Turbo controller energized until after lunch to allow rotor to slow down after braking.
1457 - 1500 hrs. local -> In and out of LVEA to de-energize Vertex Turbo controller
This information is based on the cumulative spectrum of recent lock stretches (9/18-9/26, computed from Fscan SFTs with spec_avg_long), plus Fscan magnetometer data.
0.997698 Hz
1.0 Hz
0.5 Hz odd harmonics (AKA 1 Hz with 0.5 Hz offset)
1.0 Hz with 0.998889 offset (new)
0.987987 Hz (new)
Attached plots:
Minor edit: offset on 4th comb mentioned has too many significant figures. Should be rounded to ~0.9989 (+/- 0.0005).
Update on the 0.996798 Hz: this comb appears in magnetometer channels between Feb 3rd and 4th*. After looking at the alogs for surrounding date range, Robert suspected the HWS as a potential source. We checked TCS-ITMX_HWS_RCXCLINKSWITCH and saw a switch on, which corresponded neatly with the date of the comb's appearance in magnetometer channels. Robert has also previously noted near-1Hz combs associated with the HWS in the CS. Richard says that the HWS can be placed on a separate power supply in the near future, which will hopefully clear this up.
*edit, typed wrong date
Saturations Cleared this morning per FAMIS #7073. The counts seen are listed below:
Fil Marc Keita Daniel
We looked at the "broken" channel 5 today, but no problem was found. After reconnecting the ISS chassis and reshuffling some of the cables under HAM2, everything was working fine again.
We took the opportunity to measure dark noise with the PSL shut off. The attached plot shows the dark noise with no light on the photodetectors and calibrated in relative intensity noise for 50 W input. Also, shown is the shot noise for 30 mA of photocurrent at the 3 x 10–9 level.
There is an AC coupling in the RIN_INNER and RIN_OUTER channels which explains the discrepancies below 0.3 Hz. ADC noise explains the difference between these channels and the RIN_ERR1 and RIN_ERR2 at frequencies of 300 Hz and above.
We are not sure why there is a difference between PDA and PDB. The PDB readout seems to be ADC noise limited through the entire freqyuency band.
I removed the drivers S1600266 and 267, added capacitor 4.7pF in the C25 position and returned the drivers to service. HV was turned back on and confirmed. See Patrick's aLog entry.
Work Permit 6175
High voltage supplies at CS, EX and EY are back on. (Filiberto EX, Patrick EY, Ed CS)
WP 6184 Added EPICS alarm levels to the Inficon BPG402 gauges. I didn't realize I didn't need to do h0vacly until after I did (no BPG402 gauges there). I burtrestored to 8am this morning.
Also took the opportunity to update the medm screens on all of the vacuum Beckhoff computers.
Jenne, Sheila
Today we wanted to get the SRC alingment under control, mainly motivated by the observation that both our noise couplings and noise change a lot over time (within and between locks) (see 29984)
SRC ASC
We have been able to close ASC loops for the SRC at full power, both pit and yaw. We used to use AS_C to feedback to both SR2 and SRM, (SRC2), tonight we are sending this signal only to SRM, and we are using ASA45I to control SR3 angle. In the past we had a cage servo runnning for SR3 pit, but this is now disabled in full lock. I used a gain of +100 for pit and +30 for yaw, but both of these loops are verry slow as is and should probably have their gains increased a lot. (input and output matrix elemetns are all positive as well). I added some code to the guardian to use this in SRM)HIGH_POWER_ASC, but we should probably try it at 2 Watts, increase the gain and check the loop shaping, and try to power up with this configuration. I'm levaing the IFO undisturbed, so hopefully we can see if our noise is any more stable tonight.
REFL WFS detour
Since we were interested in using REFL 45 WFS as signals for SRC alignment, we wanted to prevent the 200 mHz PIT oscialltion which has dominated all REFL ASC for many months now. It turns out that even without this oscialltion, we didn't see a good signal for SR3 in these WFS, but prevetning the oscialltion seems to have helped stabilize carrier power in the interferometer.
Fixed oscilations in REFL DC centering loops
I noticed earlier that the oscialtion around 200 mHz that is present in all of our REFL WFS signals seems to come from the DCP1 loop. Jenne and I measured the loop, and found that indeed it had not much phase margin at all and a ugf around 200 mHz. The only alog we found about these loops is 9966 (from 2014). It seems that since then, someone added rather aggresive 0.9 Hz low passes that ate too much phase for all of the REFL centering loops.
We replaced the filter design for these loops with the filters used for AS centering, which allow us to get bandwidths of 2.5 Hz with about 30 degrees of phase. We also copied over the 20Hz low pass, but haven't engaged it.
The attached screenshot shows the new loop shapes, the second shows the settings used when we measured these. We do not understand why DC2 (which used REFL B DC for a combination of the 2 tip tilts) seems to fall as 1/f^4 after the resonsance.
Increased gain of PRC1 P
With the DC centering loops no longer oscillating, the refl WFS signals were slightly better but still moved alot at microseism frequencies. We found that we could improve the phase margin in the PRC1 P loop, and reduce these fluctuations, by increaseing the gain in this loop by a factor fo 4 (digital gain of 60). This was what we had set the gain to in February, (see measurement attached to 26225), but it has slowly been decreased since then. The third attached screenshot shows the refl WFS pitch signals, the PRC1 gain was changed at -10 minutes.
Lisa, Sheila
Since yesterday afternoon, we have struggled with locklosses that happen 5 to 15 minutes after we power up.
There are a dramatic, very short, glitches in DRMI LSC channels. It also shows up in the ASC channels, but after LSC.
Some lockloss times are:
2016-09-27 06:40:33 UTC
2016-09-27 05:39:35
Some times (within a few seconds) when saw the glitch but we did not loose lock are:
1158999031
1158998954
1158999031
As a test to see what the problem could be, I commented out the lines in REDUCE_MODULATION_DEPTH where the POP9 to SRCL matrix element is changed. With this matrix element out, I didn't see these glitches, but the problem did seem intermittent so this could be a coincidence.
Kiwamu, Sudarshan, Jenne, Darkhan
Overview
EPICS records that are used for calculating DARM time-dependent parameters ("kappas"), were updated using corrected DARM model (with the correct sign of the ETMY_L3_DRIVEALIGN_L2L gain). These EPICS values result in reasonable kappa values (see details).
"512 Hz DAQ downsampling" filter was installed into CAL-CS synched oscillator that replicates 35.9 Hz cal. line (ESD).
Investigations showed that the synched oscillators for 35.9 Hz cal. line were running at 180 degrees out of phase w.r.t. each other. They got synched to the same phase after I played some with the oscillator settings in CAL-CS model.
Jenne noticed that today fC was oscillating between 320 and 360 Hz at the time-scale of ~20s. This issue was resolved by turning on low-pass filters in the CAL-CS model.
Details
Sudarshan confirmed that kappas calculated from SLM tool data using these EPICS values are within reasonable ranges. After updating EPICS records one of the issues was that κTST calculated in the front-end was around -1.0. Further investigations showed that the synched oscillators for 35.9 Hz cal. line in SUS-ETMY and CAL-CS models were running at 180 degrees out of phase w.r.t. each other. We could get rid of the discrepancy by setting the phase of the CAL-CS oscillator to 180 degrees (see attached plot).
After changing settings on the synchronized oscillators their phases somehow got synchronized. So, I removed 180 degrees of an additional phase in the CAL-CS oscillator. It is still not clear what was the cause for the phase of two synched oscillators being exactly 180 degrees off. Now the oscillator outputs (after the 512 Hz DAQ downsampling) are pretty much the same (TF measurement at 35.9 Hz is attached).
New EPICS values and corresponding logs were commited to calibration SVN. The values were accepted in SDF_OVERVIEW.
Tagging CDS so they can help address the bug in the synchronized oscillators!
Title: 09/26/2016, Evening Shift 23:00 – 07:00 (16:00 - 00:00) All times in UTC (PT) State of H1: Almost back to DC readout Commissioning: 0:30 MarkP back from MY 1:30 Robert to HAM6 to plug in an accelerometer, OMC was able to lock while he was in the area
Since we seem to be needing to help the BS's alignment after locklosses, and if left alone the BS drifts in pitch for many minutes after a lockloss from high power, I've modified the filters in the M1 filter bank such that the pitch ASC values get cleared slowly, rather than instantaneously.
Rather than having a pure integrator, I've added a zero to the pole at 0Hz, and another filter with a pole at that zero's freq. This way, we can toggle the first filter module, and it'll effectively slowly clear the history of the integrator. Hopefully this will make re-acquiring PRMI / DRMI easier - operators should let me know if they think it helps or hurts relative to the last week or so.
Atleast half of the shift had H1 at Nominal Low Noise!
Day's Activities:
Locking Notes:
After the long lock this morning, H1 was stuck at CHECK_IR. Adjusted COMM OFFSET, but could only get the X-arm up to 0.5; during one attempt the XARM gain was oddly at 0(!). Opted for an Initial Alignment.
INITIAL ALIGNMENT notes:
At ALIGN_PRM, AS_AIR video looked ugly & would not lock up. Errantly moved PRM & PR2 to a time during the long lock....but this cancels the INPUT_ALIGN pointing we had (!). Kiwamu returned PR2 to it's aligned value and then continued with Alignment.
Ops Locking Note: (OpsInfo!)
I didn't get to all the alogs over the weekend. Jenne filled me in on a new Guardian state (CHECK_MICH_FRINGES) for those times when you are locking DRMI and alignment looks bad/ugly & when there is hardly any flashes. CHECK_MICH_FRINGES allows you to optimize your BS alignment for DRMI locking. Please see Jenne's alog & also the new entry in the Ops Sticky Note wiki.
Trouble with MODE17 (ITMX 15542 Hz) could be due to instability from large phase changes with slight frequency drift in our damping loop. Mechanical mode frequency at beginning of a cold lock sits around 15541.875 Hz and has drifted to 15541.95 Hz after about three hours. The previous bandpass filter carried ~40 deg phase change over that time. In an effort to fix this, I've made two BP filters that are shifted versions of one another to be turned on as the frequency migrates. The guardian uses the PLL frequency monitor to track the average frequency change of the mode peak and make the filter change switch at the appropriate time. Fall offs and notches in the filters are such that center of BP --> switching point ~10 deg. I've watched the guardian successfuly make this change.
Proof of concept appears to have worked; Mode17 was kept stable throughout frequency drift well beyond the previous 3 hourish mark. Lockloss occured after ~6 hours from the partner Mode25 who saw a phase change of ~140 deg over this time, eventually probably driving up this usually easily dampable mode. Attachment shows freq drift over last night's lock. I've added stepping bandpasses to both modes (controlled with the guardian) now to accomodate frequency drift over much longer locks.
Sheila, Keita
The 3rd whitening filter does not switch for AS_C segment 3. The readbacks look OK.
Last Tuesday we pulled the chassis and verified everything worked. This wee we pulled the chassis and verified everything worked. Finally while watching the binary switches we were able to trace it down to a pin in the cable pulling out of its socket. This was very troublesome because it would work as long as we had a breakout board inserted in between the cable and the chassis. Watching the signals while re-assembly took place we per able to narrow it down and found the problem. We have removed the back-shell of the connector and shoved the pin back in place. Next Tuesday we will crimp on a new pin. So we have not closed the work permit yet.
Work Permit 6180
Summary: Repeating the Pcal timing signals measurements made at LHO (aLOG 28942) and LLO (aLOG 27207) with more test point channels in the 65k IOP model, we now have a more complete picture of the Pcal timing signals and where there are time delays. Bottom line: 61 usec delay from user model (16 kHz) to IOP model (65 kHz); no delay from IOP model to user model; 7.5 usec zero-order-hold delay in the DAC; and 61 usec delay in the DAC or the ADC or a combination of the two. Unfortunately, we cannot determine from these measurements on which of the ADC or DAC has the delay. Details: I turned off the nominal high frequency Pcal x-arm excitation and the CW injections for the duration of this measurement. I injected a 960 Hz sine wave, 5000 counts amplitude in H1:CAL-PCALX_SWEPT_SINE_EXC. Then I made transfer function measurements from H1:IOP-ISC_EX_ADC_DT_OUT to H1:CAL-PCALX_DAC_FILT_DTONE_IN1, H1:IOP-ISC_EX_MADC0_TP_CH30 to H1:CAL-PCALX_DAC_NONFILT_DTONE_IN1, and H1:CAL-PCALX_SWEPT_SINE_OUT to H1:CAL-PCALX_TX_PD_VOLTS_IN1, as well as points in between (see attached diagram, and plots) The measurements match the expectation, except there is one confusing point: the transfer function H1:IOP-ISC_EX_MADC0_TP_CH30 to H1:CAL-PCALX_DAC_NONFILT_DTONE_IN1 does not see the 7.5 usec zero-order-hold DAC delay. Why? There is a 61 usec delay from just after the digital AI and just before the digital AA (after accounting for the known phase loss by the DAC zero-order-hold, and the analog AI and AA filters). From these measurements, we cannot determine if the delay is in the ADC or DAC or a combination of both. For now, we have timing documentation such as LIGO-G-1501195 to suggest that there are 3 IOP clock cycles delay in the DAC and 1 IOP clock cycle delay at the ADC. It is important to note that there is no delay in the channels measured in the user model acquired by the ADC. In addition, the measurements show that there is a 61 usec delay when going from the user model to the IOP model. All this being said, I'm still a little confused from various other timing measurements. See, for example, LLO aLOG 22227 and LHO aLOG 22117. I'll need a little time to digest this and try to reconcile the different results.
By looking at the phase of the DuoTone signals we can constrain whether there is any delay in ADC side (like Keita's analysis here). The DuoTone signals are desgined such that the two sinusoidal signals 960 Hz and 961 Hz will be maximum at the start of a GPS second (and also in phase with each other). To be presice, the maximum will be 6.7 µs delayed from the integer GPS boundary (T1500513). The phase of 960 Hz signal at IOP (L1:IOP-ISC_EX_ADC_DT_OUT) is -92.52 degrees with respect to GPS integer boundary (LLO a-log 27207). Since the DuoTone signal is supposed to be maximum at GPS integer boundary i.e, it is a cosine function, this corresponds to -2.52 degrees (estimate of 92.52 assumes it is a sine function) phase change. Converting this phase change to time delay we get 7.3 µs. Since there is an inherent 6.7µs delay by the time the DuoTone signals reaches the ADC, we are left with only 0.6 µs delay possibly from ADC process (or some small systematic we haven't accounted for yet). This is what Keita's measurements were showing. Combing this measurment and above transfer function measurments we can say that we understand the ADC chain and there are no time delays more than 0.6 µ in that chain. This also suggest that the 61 µs delay we see in ADC-DAC combination exist completely in DAC side.
The DuoTone signals are sine waves, so a minor correction to Shivaraj's comment above, the zero-crossing corresponds to the supposed GPS integer second. I looked at a time series and observe that the zero-crossing occurs at ~7.2 usec. Since the analog DuoTone signal lags behind the GPS second by ~6.7 usec, I can confirm that the ADC side has essentially no delay. Thus, the 61 usec seen through the DAC-ADC loop is entirely on the DAC side. Attached is a time series zoom showing the zero crossing of the DuoTone signal.
When using dtt to make a transfer function measurement between an IOP model and a user model, one has to keep in mind that dtt does another decimation silently. This is due to dtt trying to match the number of data points between two models. Fortunately, this does not seem to affect the phase, see my note at https://dcc.ligo.org/T1600454.
Updated the timing diagram for consistency with other timing measurements (LHO aLOG 30965). See attached PDF to this comment.
I've gone over the spectra during this time and Corey did turn the mode around in time; it was successfully damping down (and well below usual lockloss amplitudes) when we lost lock probably for unrelated reasons. Thanks TJ for the successful alert system and Corey for the quick response.
Mode rang up in the first place due to phase drift; I had set up guardian controlled filter changes to account for this but there was an error that put the SUS_PI node down between the two locks last night/this morning, so filter switch never happened. I've since fixed this.