Evan, Daniel
17:12:30 UTC Oct 7 2016:
17:16:30 UTC Oct 7 2016:
17:18:30 UTC Oct 7 2016:
17:24:30 UTC Oct 7 2016:
17:32:00 UTC Oct 7 2016:
17:34:30 UTC Oct 7 2016:
18:06:30 UTC Oct 7 2016:
SEI: All OK – Working on ESs sensor correction SUS: No Report CDS Ele: All OK CDS Swt: All OK – Fixed problem with bad commissioning frame effecting NDS PSL: All OK – PSL has been running since the Diode controller swap TCS: No Report VAC: All OK – Kyle planning for the 5 day RGA bake out at End-X
There was only a short, not very good lock, after the DBB signal started to be acquired. I used ten minutes of data from GPS 1159862777 to compute the coherence between DARM (CAL-DELTAL) and the DBB QPD signals.
The first plot shows that the most coherent signal is Q1Y.
The second plot shows the transfer function from Q1Y (in arbitrary units, not sure about the calibration) to CAL-DELTAL (properly calibrated in meters, including the de-whitening filter). The shape is quite smooth, and it looks like a monothonic increase like f for most of the range. The low frequency noise of the IFO was quite bad, so I could only measure coherence above 100 Hz. Hopefully this will be improved in future locks.
However, I could fit the measured transfer function between Q1Y and CAL-DELTAL with a 3rd order model. The fit is shown in the third plot: the result is reasonably good.
I then converted the model into a IRR filter and computed the time-domain subtraction of the Q1Y signal from DARM. The result is quite good, most of the bump has disappeared, see the 4th plot for a comparison of the spectra and the 5th plot for the residual (basically null) coherence of the subtracted DARM with DBB signals.
Of course, we'll have to check in future locks how much this coupling changes over time and how hard it is to compensate for this change.
Trying to check where the laser power went to. From the attached plots, the drop in laser power is not easily explained by a power drop in the laser diodes. So my suspicion is that there's a small resonator alignment change. Perhaps not all that surprising given the temperature change of the laser due to being off for a few days.
Just a quick report. Broadband noise in 200-1000 Hz was also visible at a mid-high input power of 27 W.
I did not insert the cutoff filters in the hard ASC loops. ISS 2nd loop was fully engaged with a gain of 19 dB and the boost on. TCS was held at the lock acquisition settings, i.e. [CO2X CO2Y] = [500 mW 1000 mW]. The period when the interferometer was low-ish noise is in 8:15 - 8:25 UTC which was followed by lockloss due to me failing to handle PI mode 27.
Synopsis -- Not surprisingly, the broadband noise in terms of RIN at the DCPDs seems to grow proportionally to the carrier field amplitude in the interferometer.
[Cross correlated noise in DCPDs]
Here is a plot of cross-correlated noise between OMC DCPDs A and B. DARM loop suppression is removed by a post process.
For comparison, I overlaid the cross correlated spectrum of a 50 W lock which is from Sep 30th (30115). For both data, the OMC DCPD sum current was held at 20 mA by the DARM loop. It is clear that the mid-power lock (27 W in blue) has a slightly lower noise level in 200 Hz - 2 kHz. The high power lock has a much higher noise level above 2 kHz. I attach the plot in fig format as well. The pcal calibration line at 331.9 Hz for the mid-power lock was smaller than the high power lock by 20 %. Qualitatively, this is due to the higher optical gain in the high power lock although the ratio ideally should be sqrt( 27 W / 50 W ) = 36% instead of 20%. Probably this discrepancy can be partly due to the smaller recycling gain for the high power lock.
Now, if one scales the mid-power spectrum so that both the pcal lines have a same height at 331.9 Hz, it gives you the following plot.
The two curves overlap more in 200 Hz- a few kHz. This simply means that the broadband noise scales with the field amplitude of the carrier light circulating the arm cavities. Because the DARM optical gain also scales with the carrier field amplitude in the same way, this unfortunately means that the calibrated displacement noise does not change regardless of the laser power level. This rules out some local electronics pickup/cross-talks, but does not rule out laser noise couplings (jitter, intensity, frequency) or displacement noise.
Terra, Travis, Kiwamu,
Here are some unexpected issues that we addressed tonight.
The ASC DC SUM channels have been split into to SUM and NSUM. The SUM channel has a steep low pass, so it can be used to normalize PIT and YAW. NSUM is the new fast channel. See alog 30214.
Thanks, Daniel. I have just edited the FAST_SHUTTER guardian so that it looks at AS_A(B)_DC_*NSUM_OUTPUT* instead of *SUM_OUTPUT*. Also, I found my original statement in the above entry a bit wrong. AS_A(B)_DC_SUM did not have a low pass at all before Oct.5th as Jenne reported in 30214.
The FAST_SHUTTER is now at rev14405.
TITLE: 10/07 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Lock Aquisition
INCOMING OPERATOR: None
SHIFT SUMMARY: Started shift with an initial alignment. After IA, spent the entire shift working with Jenne and Kiwamu diagnosing issues with ASC loops and fast shutter. Only made it past DRMI a few times.
LOG:
5:30 Filled TCSY chiller
6:00 Shift end due to early start for 3IFO meeting
Kiwamu, Matt E, Terra
We broke our overnight lock this morning from Mode2 (ITMX 15520.7 Hz) ringing up; usually this is easily damped back down with a slight phase change but there was no operator in the chair at the time. This mode has been especially prone to ring up with very minor (a few degrees) phase drifts as the frequency drifts over ~3 hours lock across a static bandpass.
I measured this mode's open loop transfer function last night (with the same settings it later rang up with). Width of loop is 24 mHz; cursors at UGFs shown on transfer function plots. Measured Q of mode is ~20 M, giving a half-width of 0.776 so the gain on resonance of our loop is about 31. Phase differential across this fairly wide frequency range is ~160 deg, so we're close to instability.
To this end, I've halved the gain of the damping drive. I also shifted the bandpass filter: previously the resonant frequency range was sitting slightly off center picking up ~30 deg and now it shifts around within ~2 deg phase window (for a five hour lock at least). We had trouble locking tonight and are now working at low power, so I'll have to remeasure another time.
Joe B, Darkhan T,
From the past day's lock stretches it seems that now the DARM time-dependent parameters (kappas) calculated in the CAL-CS model are mostly within expected ranges. The optical gain seems 10% lower compared to what the DARM model suggests.
Related alogs (kappas from September): 30219, 29992.
Cal. line coherence averaging:
Coherence calculations in the CAL-CS front-end model outputs overestimated coherences for the calibration lines between Sept. 20 and now. The issue will be fixed by replacing the averaging C code, BUFFER_AND_AVERAGE.c with a previous version (modification to the script proposed in LHO alog 29744 will be reverted).
The first figure shows the results of 2.5 cycles of crystal chiller total flow variation (24.2 l/m vs. 22.2 l/m) in DARM and in the 1QX signal of the DBB quad diodes. The diode signal shows a detectable increase with flow from about 60 to about 6000 Hz, with the biggest increase in the 1000 Hz region. DARM increased from about 200 to 2000 Hz, with the biggest increase around 450 Hz.
The second plot shows other DBB diode signals. The peaks do not change as much as the rest of the jitter spectrum (see 1000 Hz). The lower percentage changes at some of the peaks are more consistent with the changes in table accelerometer signals. The water signal is apparently a less dominant driver of the peaks than of the smooth regions, and the source of the peaks may therefore be easier to access and damp should need be.
Robert, Jason
Jonathan, Jim, Dan, Dave
Jim W reported data errors from 06:14PDT this morning in the full frame. We found that the commissioning frame file for this time is correct on the LDAS QFS file system, but corrupt when NFS exported by the h1ldasgw2 NFS server machine to the nds machines. This is a relatively new solaris server installed this summer to take read-only exports away from h1ldasgw0 when h1fw0 was unstable.
I've opened an FRS for this #6370
Investigation is continuing, including a full characterization of the problem.
I have a new build of daqd frame writer running h1fw2. Now when we write raw frame files we also write a checksum file next to the file. This will give us a view of what the daqd says was written out so that LDAS/DCS can verify they receive the file CDS produces. Tomorrow I will do this on the trend files, and reflect some of this information into EPICS so that we can get a graphical alert when frame writers produce different output.
I now have a build of daqd frame writer running on h1fw2 (and the test stand) that provides checksum files for all frames being written.
In addition it adds four EPICS channels that give 32bits of the checksum to help with medm monitors for Dave.
Prefix each with IFO:DAQ-FW[012]_ (in general, currently only deployed for H1:DAQ-FW2_)
FRAME_CHECK_SUM_TRUNC
SCIENCE_FRAME_CHECK_SUM_TRUNC
SECOND_FRAME_CHECK_SUM_TRUNC
MINUTE_FRAME_CHECK_SUM_TRUNC
Dave, Nutsinee
There are still some issues (several white channels on the medm screen, unclear where the files are being written) but so far the new code runs on Ubuntu 14 at both end stations without complains.
This is a low level sanity check and a part of the recent delay study (e.g. 29259). I have measured a transfer function between DARM2_OUT and SUS-ETMY_L3_ISCINF_IN1 using dtt while the interferometer was locked last night. The measurement agrees with 1 cycle user model delay (=61 usec). See the attached.
Here is another low level sanity check. The attached shows a tranfer function of signals from the SUSETMY user model to its associated IOP model. It shows a 1 user model clock delay (61 usec) as expected. There is a large deviation above 7 kHz which I don't know why. The FIR filter (T1600454) was included in my 'expected' model.
WP6220 Inclusion of DBB channels to DAQ to monitor jitter of HPO
Daniel, Dave:
a new h1psldbb model was installed and the DAQ was restarted this afternoon (14:50 PDT). Daniel decided that this version initially write its fast channels to the commissioning frame only while the ECR for science frame inclusion was being processed. Daniel's ECR (E1600301, FRS6386) has been approved and these channels will go into the science frame on the next DAQ restart.
Jenne, Evan
We looked again at RFAM on the 9 MHz REFL readout.
The attachment shows REFL9I (and REFL LF) for four different PD powers. Additionally, at the fourth power we locked the ISS outer loop (dc coupled with boosts) just to check that the RFAM does not change. The REFL centering loops were on the whole time.
At no power were we able to see anything that looked shot-noise limited, so we were not able to independently check the transimpedance and demod gain for REFL9I. However, the limit placed by the high-frequency noise already seems to indicate that we are missing some gain in the signal chain. Thus far I have been using 2900 V/W for the PD transimpedance plus demod TF, but this would produce spectra that are below the expected shot noise level.
I double-checked the schematics for the REFL LF path and found that the current digital calbration from counts back into watts seems OK. The PD powers given in the attachment are based on this LF calibration.
Finally, note that the dark noise at several kilohertz is larger than the noise when the PD has power on it. No clue about why.
The attachment shows REFL9 and REFL45 signals (in ct/rtHz), with the colors corresponding to the powers given in the previous attachment.
REFL9Q appears to be shot noise limited above 100 Hz, since the noise grows with the square root of the power.
In comparison, REFL9I looks fishy. Its dark noise is higher than REFL9Q, and above 500 Hz shows a noise that is lower than the REFL9Q shot noise.
We are tempted to say that the analog Q channel of the REFL 9 demod board is broken (recall that analog Q corresponds to digital I, and vice versa), or the corresponding whitening channel is broken. (However, since analog Q is teed off and hooked up to the summing node board, it's possible that there's some strange interaction with the SNB.)
Anyway, if we use REFL9Q to calibrate the rf gain of REFL9, this implies a total gain of 2.88e6 ct/W (= 4.4e-4 ct/rtHz / sqrt(4*h*nu*P0), with P0 = 29 mW). We can refer this back to the demod board output by undoing the digital gains (0.18 ct/ct), the ADC conversion (2^16 ct / 40 V), and the whitening gain (12 dB) to arrive at an rf gain of 2600 V/W, which is not too far off from the old value of 2900 V/W.
(Corey, Jason on phone)
Last night, Nutsinee came in early for her OWL shift & just before midnight the PSL tripped. We opted to leave it tripped for the night to wait for Jason to walk us through recovery procedure. Atleast for now, we always want to run through this procedure with one of our PSL people (Jason or Peter K). Here are my rough notes:
Now on to "Aligning" on Observatory Mode!
PSL Status NOTE:
The Xtal chiller has tripped 3 times in the last week:
then 4 days later:
now 3 days later:
The last two events imply an overpressure condition. The problem seems more significant than replacing lost fluid from a slow leak.
Was the cap which "blew off" in today's event the "bleeding cap" (section 4.2.2), the "filter sleeve cap" (section 6.2) or the "filler pipe cap" (section 6.3 of T1100374-v1, "200 W Laser Crystal Chiller Manual")?
BTW, LRA = Long Range Actuator; see section 7.2 of T0900641-v5, "Under Manual 200 W Laser".
From my quick look through the alog I think the PSL has tripped more than this (for LHO to concur). From what I can see it has gone off 11 times in the last week (thats as far as I looked back).
I'm not sure if all the same problems, but alogs reporting the PSL laser off are LHO alogs:
10/2/2016
LHO alog 30160
LHO alog 30154
10/1/2016
LHO alog 30146
LHO alog 30143
9/30/2016
LHO alog 30118
LHO alog 30086 (due to power glitch)
9/29/2016
LHO alog 30076
LHO alog 30063 (this alog reports two different instances of the laser going off)
9/27/2016
LHO alog 30016
9/25/2016
LHO alog 29964
The filler pipe cap popping out is a known thing (happens all the time at LLO) when the chiller turns OFF. At LLO at least this has not shown to be due to any problem (just a consequence for whatever reason when the chiller is turned off). Its why we try to not turn these chillers OFF if can help it as they "burp" water over the floor and pop these fill caps even when trying to restart
I have as one of the main agenda items of this Wednesdays PSL meeting to discuss this problem and see if we can work out whats going on. Some statistics on how many times happened in say the last month or two, what the PSL trip was attributed to, and how many times happened before and after the chiller swap (to see if accelerating or at the same rate), would help this. Im not sure if FRS fault reports have been made for each laser trip to make this search easy for us remote to the site to do and work out how much lost observatory time we have had due to this issue.
Yes, Dennis, as Matt says it's the Filler Pipe Cap that pops off. We have had (3) trips since Friday evening (so Matt is probably right about there being more trips over the whole week). OH, and I should correct ourselves here because we just had a 4th Weekend PSL trip (this was just after I had H1 at NLN for 15min. This time the cap was blown off and there was a puddle on the floor.
Able to get back in 30min this time (vs 60min this morning).
OK, back to locking.
Another Note:
Something I wanted to add about the chiller was that when filling it, I noticed quite a bit of turbulence in the fill pipe. And you could see a air bubble vortex/tornado in there. Something we probably don't want if air bubbles are postulated as a trigger for flow sensor trips.
We see these bubbles as well in the chiller fill pipe at LLO with no chiller trips due to it (hopefully haven't jinxed myself). Perhaps post a movie of it so can see if looks the same as here
Jason & crew will be investigaing tomorrow. We should ask them to record a video of it.
Movie of the water turbulence of LLO's crystal chiller fill tube posted at LLO alog 28397
I took a video, but I can't post it; my phone only takes video in .mp4 format, which is apparently not a valid file type for upload to the alog. Huh.
I attached a still from the video to give you some idea of what we've been seeing here for the last few weeks. It's appearing to fluctuate though; the video was taken on Monday, 10/3/2016, but this morning our fill port looks very similar to what's seen in Matt's video of the LLO crystal chiller fill port.
Spectra attached.
Coherence (modulation on)
Using 2600 V/W for the demod gain and transimpedance, and 29 mW of dc PD power, this implies the following AM depths:
Using 0.22 rad and 0.28 rad for the 9 MHz and 45 MHz modulation depths, this implies the following AM/PM ratios:
The attachment contains a budget of the expected CARM residual. The in-loop error point is taken from the CM board control signal, as was done previously. Here I used 2600 V/W for the transimpedance and demod gain.
The other measured traces are taken from the REFL9I readback (not from the CM board), so in principle there could be some extra dark noise at the error point from the summing node board or CM board. However, based on the O1 level this is of the same order as the shot noise (so we are not missing a huge amount of extra noise in this estimate).
Attaching earlier RAM plot, this time with informative labels
Here is a time series of REFL LF during the modulation depth reductions that happen during lock acquistion.
During the 9 MHz depth reduction (from 0.22 rad to 0.11 rad), the dc power changes from 4.83(3) mW to 4.27(3) mW. That means that after the modulation depth reduction, 4.08(4) mW of the dc light is from the carrier and 0.19(2) mW of the dc light is from the 9 MHz sideband (this assumes the 45 MHz contribution is negligible).
Note that the dc level is still settling to its final value of ~3.7 mW, so it's possible that these power ratios are evolving during the lock.