J. Kissel, T. Sadecki, E. Merilh

We're find that all of the QUADs are saturating much more than usual this morning. Tracing the large control signals, it looks like that HARD loops' error signals are much larger than they were during the -12 hours ago lock stretch, and have been increasing since half way through. The investigation is still preliminary, but we've lost two locks already to this excess noise. Note that this is all well before any of the ISC_LOCK guardian's LOWNOISE_ASC step of which Jenne told us to be mindful (see LHO aLOG 28109).

I have a suspicion it's something obvious to the ISC experts, but this is what the engineering team gets for trying to replicate such rapid improvement from late last night!

The investigation continues...

Saturations start happening as soon as the CHARD Loops come on in ENGAGE_SRC_ASC.

Added 1296 channels. Removed 589 channels. (changes attached)

yesterday fw0 went very unstable for a few hours. We noticed that during this period file access on the QFS server (h1ldasgw0) was normal, but even simple file actions like a long listing (ls -al) from h1fw0 could take many seconds. Rebooting and power cycling both machines did not fix this. We configured h1fw0 to only write the science frame, which did make it more stable. At this point the CRC epics channels for fw0 and fw1 were different, but the two science frames written were byte identical. It looks like the CRC is the checksum of the data block the frame writer needs, if it is not writing the commissioning frame then it is the CRC of the science data only. At this point we also did  full DAQ restart (17:37PDT) to resync everything. At this point we felt that having the CRCs match is an important diagnostic, so h1fw0 was reverted to write both frames again, and this time it went stable. 

h1fw0 has been stable since the 17:37PDT DAQ restart (we don't know why). h1fw1 has restarted 3 times since then (01:04, 01:32, 07:03 PDT Friday 7/1). I think this level of fw1 instability is acceptible if fw0 continues to be 100% stable.

Our investigation suggests that Keith's suggestion of upgrading the NFS link between FW and QFS-NFS server from 1GE to 10GE would be a good test. Dan has some fiber-optics 10GE cards we could borrow. We will schedule this test after ER9 or before if the instability reappears.

J. Kissel 

Since Jenne was able to get a beautifully long 6+ hour lock stretch last night (see LHO aLOG 28109), I've moved the PCALX high frequency calibration line frequency up to 2501.3 Hz. I've also increased the amplitude of the line to 40 000 [ct] of excitation in H1:CAL-PCALX_PCALOSC1_OSC_SINGAIN. 

I'm not *sure* it matters (since Kiwamu just recently updated the OMC DCPD whitening filter compensation; see LHO aLOG 28087), but let's be mindful that Jenne had added a stage of OMC whitening during that long lock stretch. It looks like she turned on the whitening right when the inspiral range jumps from just under 40 to just under 50 Mpc, so we should just consider the whole lock stretch having the first stage of whitening ON. 

I've accepted the new frequency (and increased amplitude) in SDF under the OBSERVE.snap file.

Jenne, Carl, Ross

We have been testing some of the filters settings for PI mode damping. During the lock tonight we saw 5 unstable modes at 15009Hz (ETMY), 15522Hz (ITMX), 15541Hz (ETMX) , 15542 (ETMY) and 18038Hz (ETMY). The 15522, 15541 and 15542 modes are all unstable after four hours of the 40 W lock. The other two are unstable temporarily as the thermal transient passes through them.

We have demonstrated that we can get steady state damping of the three unstable modes using either 10 Hz band pass filters or iwave tracking (first plot shows damped state amplitudes). We will leave the it in the 10 hz band pass filter configuration. We also show that we can excite and damp the steady state mode at 15009Hz using a 10 Hz bandpass filter (second plot) which should result in a reduced mode amplitude during the transient. 

We will put these filter settings into the guardian. The ETMX damping will not happen unless the SUS_PI guardian state is manually set to ETMX PI DAMPING.

Between 11 to 12 UTC all four ring heaters were stepped by 0.5W for approximately 15 minutes for test mass mode identification.

I lowered the gain of the ASC HARD loops pretty significantly, and that has drastically improved our DARM sensitivity. 

It's starting to be hard to see where the coherence is coming from, but it looks like it's not really any of the ASC loops above 10 Hz.  (The coherences plots in the attachment were taken while A2L was running, so the 8 lines around 20Hz are ignore-able.) 

I ran A2L again, and there weren't any big changes, or improvements to the sensitivity. 

I turned off the MICH and SRCL feedforward for a while, so that I could see about re-tuning the FF filters.  I did SRCL injections starting at 09:00 UTC, MICH injections starting at 9:14 UTC, and PRCL injections starting at 9:23 UTC. 

Attached is a screenshot of my current HARD loop settings.  I have put them into guardian in the new LOWNOISE_ASC state that happens just before we switch to the lownoise ETMY ESD, but have not run the new code for this state yet.  Be mindful of this when going to lownoise tomorrow.

Other than PI damping work ongoing, the IFO is undisturbed, starting at 9:40 UTC.

Oh, and I know that we decided to run ER9 without OMC DCPD whitening in case the PIs got too big, but since they're okay right now, I have the whitening on so we can see the pretty new shot noise level that we have.

TITLE: 07/01 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Lock Aquisition
INCOMING OPERATOR: None
SHIFT SUMMARY: Eventually got a solid 2.5hr lock in "low noise" and Jeff was able to finish up his measurements. Before that we had two lock losses caused by bounce modes very quickly runging up, although one was exactly 9.80Hz which falls between our other known bounce modes. PI was the cause of the long lock, ITMX. Locking has been the same as it has the past few days (AS I typed that we started having issues getting through turning on the ASC).
LOG:

• 00:36 - DAQ restart to get frame writers to hopefully sync up a bit.
   

J. Kissel, J. Driggers, T. Shaffer, D. Tuyenbayev, E. Goetz, K. Izumi

Over two ~1-2 hour lock stretches, I've managed to get the measurements needed for a baseline calibration update for ER9. Complete success! We'll work on the data analysis tomorrow, but I list the locations where all measurements have been committed to the CAL repo below. 

Of particular notes for the configuration of the IFO while doing these measurements:
- The OMC DCPDs have *no* stages of whitening employed. We've decreed that given the unknown success rate of PI damping over the next few days, it's more robust to leave the whitening off. We're not gaining too much in the high frequency end of the sensitivity anyways.
- All suspensions, including ETMY, have had their PUM stage switched to "Acq ON, LP OFF," i.e. state 2, or the highest range (i.e. not low noise). After some digging, Jenne found this was changed for some reason about a month ago, and maybe a setting that got lost in the power outage or something. 
I don't think either of these seemingly detrimental configurations are all that bad for ER9, given the bigger sensitivity issues elsewere.

Also note, in order to break the correlations we've found in O1 data between measurements of Actuation Stage Strength (see e.g. LHO aLOG 28096), I've taken an independent PCAL2DARM sweep for every isolation stage.

Measurements needed for Sensing Function:
/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/PreER9/H1/Measurements/PCAL/
2016-07-01_H1_PCAL2DARMTF_4to1200Hz_SRCTuned.xml
    Exported as:
    2016-07-01_PCALY2DARMTF_4to1200Hz_A_PCALRX_B_DARMIN1_coh.txt
    2016-07-01_PCALY2DARMTF_4to1200Hz_A_PCALRX_B_DARMIN1_tf.txt

/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/PreER9/H1/Measurements/DARMOLGTFs/
2016-07-01_H1_DARM_OLGTF_4to1200Hz_SRCTuned.xml
    2016-07-01_H1_DARM_OLGTF_4to1200Hz_A_ETMYL3LOCKIN2_B_ETMYL3LOCKIN1_tf.txt
    2016-07-01_H1_DARM_OLGTF_4to1200Hz_A_ETMYL3LOCKIN2_B_ETMYL3LOCKIN1_coh.txt
    2016-07-01_H1_DARM_OLGTF_4to1200Hz_A_ETMYL3LOCKIN2_B_ETMYL3LOCKEXC_tf.txt
    2016-07-01_H1_DARM_OLGTF_4to1200Hz_A_ETMYL3LOCKIN2_B_ETMYL3LOCKEXC_coh.txt


Measurements needed for Actuation Function:
/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/PreER9/H1/Measurements/FullIFOActuatorTFs/
2016-07-01_PCALYtoDARM_FullLock_L1.xml
    2016-07-01_H1SUSETMY_PCALYtoDARM_ForL1Drive_FullLock_tf.txt
    2016-07-01_H1SUSETMY_PCALYtoDARM_ForL1Drive_FullLock_coh.txt

2016-07-01_H1SUSETMY_L1toDARM_FullLock.xml
    2016-07-01_H1SUSETMY_L1toDARM_State1_FullLock_tf.txt
    2016-07-01_H1SUSETMY_L1toDARM_State1_FullLock_coh.txt


2016-07-01_PCALYtoDARM_FullLock_L2.xml
    2016-07-01_H1SUSETMY_PCALYtoDARM_ForL2Drive_FullLock_tf.txt
    2016-07-01_H1SUSETMY_PCALYtoDARM_ForL2Drive_FullLock_coh.txt

2016-07-01_H1SUSETMY_L2toDARM_FullLock.xml
    2016-07-01_H1SUSETMY_L2toDARM_State2_FullLock_tf.txt
    2016-07-01_H1SUSETMY_L2toDARM_State2_FullLock_coh.txt


2016-07-01_PCALYtoDARM_FullLock_L3.xml
    2016-07-01_H1SUSETMY_PCALYtoDARM_ForL3Drive_FullLock_tf.txt
    2016-07-01_H1SUSETMY_PCALYtoDARM_ForL3Drive_FullLock_coh.txt

2016-07-01_H1SUSETMY_L3toDARM_LVLN_LPON_FullLock.xml
    2016-07-01_H1SUSETMY_L3toDARM_LVLN_LPON_FullLock_tf.txt
    2016-07-01_H1SUSETMY_L3toDARM_LVLN_LPON_FullLock_coh.txt

J. Kissel

After a nice long ~1 hr lock stretch at 41.5 W, I've moved the long-duration PCALX "sweep" from 1501.3 to 2001.3. I've accepted the new frequency on PCALX's SDF system (under the OBSERVE.snap file). We'll need a lock stretch of at least 2hrs for this line. Recall we're following the same plan as was done in post O1; see e.g. LHO aLOG 24843.

No clue for the mystery noise, but it's not the jitter from HPL linearly coupling to DARM.

In the attached, you see coherence between darm and usual things as well as DBB beam jitter signals measuring the HPL jitter. Nothing is obvious about mystery noise at e.g. 100Hz.

DBB PMC was locked, aligned using WFS, and WFS control was turned off. IFO was locked at nominal low noise with 40W.

We will gain something by retuning SRCL and MICH FF for f<100Hz, but these are not the main sources of the mystery noise at e.g. 100Hz. We might also benefit from pushing the CM gain to lower high kHz noise.

There's some coherence with ISS second loop sensors between 100 and 1000Hz. (Note that these ISS second loop signals are NOT what is plugged into the second loop board, they are digitally generated from indivisual PD output and that's how we still retain PD58 sum.) It's not clear if this is the intensity noise, changing ISS second loop gain might reveal something but this cannot be the mystery noise.

DBB_QPD_1DX etc. are WFS errors, and DBB_QPD_1QX etc. are WFS DC pointing. Apparently there's some jitter peak at 1kHz which shows up a bit in DARM, and there might be something at about 30-40Hz, but except these, nothing. PIT WFS signals (PSL-DBB_QPD_1DY and 2DY, these are blue and brown in the middle panel) might be showing the same broad 100-1000Hz thing that the ISS sees.

In the second attachment, I measured the IMC WFS and it shows some structures which we might have to do something about once we get rid of the mystery noise, but it doesn't look as if linear coupling of the jitter explains the mystery noise. (In this second attachment, strange bump in DARM at 300Hz is because something was injected for calibration measurement.)

[Marie, Jenne]

Marie suggested that we run A2L, which we did.  She was totally right - we should have done this a week ago!  We win about an order of magnitude in sensitivity at low frequencies.  This will make things much easier for the calibration team.

Attached is a spectrum showing the enormous improvement.  This reduced the coherence between DARM and CHARDY, but not to the low level that CHARDP is, so there is probably more that we could do to get even better sensitivity with A2L.

Edit:  The new A2L values have been saved in both the down.snap and observe.snap sdf files.

I have measured the transfer functions of the newly installed whitening chassis (alog 28010, ECR:E1600192, SN:S1101627) for the OMC DCPDs. I used LISO for fitting the data.

[1st stage for DCPD A]

Best parameter estimates:
pole0:f =  18.6776675365k +- 10.45 (0.0559%)
pole1:f =  14.3409807014k +- 6.292 (0.0439%)
pole2:f =  98.9444053052k +- 32.86 (0.0332%)
pole3:f =  10.3243596197 +- 359.4u (0.00348%)
zero0:f =  985.3423393170m +- 79.51u (0.00807%)
factor =  998.8350760502m +- 67.01u (0.00671%)

Final chi^2=0.00252952

[2nd stage for DCPD B]

Best parameter estimates:
pole0:f =  18.5698573085k +- 14.71 (0.0792%)
pole1:f =  14.6043378758k +- 9.31 (0.0637%)
pole2:f =  100.4496012696k +- 43.14 (0.0429%) > MAX
pole3:f =  10.0731025960 +- 470.5u (0.00467%)
zero0:f =  961.4535554217m +- 104.7u (0.0109%)
factor =  997.7425128879m +- 90.38u (0.00906%)

Final chi^2=0.00454374

[Small notes]

I have also measured the same transfer functions with a 4 times smaller excitation signal in order to check some sort of saturation-induced measurement error. I did not quantitatively look at these data, but plotting them on top of the above data showed excellent agreement (to my eyes).

[SVN info]

All the data, scripts and figures are checked into SVN at:

/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O2/H1/Scripts/OMCDCPDs

/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O2/H1/Measurements/OMCDCPDs/2016-06-30

/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O2/H1/Results/OMCDCPDs

Attached are the past 7 day pitch, yaw and sum trends for all active H1 Optical Levers

TITLE: 06/30 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Lock Aquisition
INCOMING OPERATOR: TJ
SHIFT SUMMARY: First half of the day was dedicated to maintenance and measurments by several parties.  Afterwards, locking has been fairly successful up to INCREASE_POWER several times, and we made it to NLN finally.
LOG:

15:33 Kiwamu and Evan G. to HAM6 OMC chassis measurements

16:00 Fil to LVEA getting serial #'s for vacuum racks and chassis

16:02 King Soft on site

16:18 Patrick updating MX vacuum Beckhoff

16:22 Richard to reinstall OMC AA chassis

16:25 Hugh to EX

16:36 Richard done

17:06 Fil done

17:08 Patrick done

17:09 Cheryl to IOT2L taking pics of beam dump

17:23 Kiwamu done, Cheryl done, Hugh done

17:39 Kyle back

17:59 DAQ restart

19:19 Richard to EX, EY, and MY

20:08 Richard back

20:37 reset ALS Y VCO

Between yesterday and today, Gerardo and I successfully bonded the ears to core optic mass ETM15, destined for LLO for post-O1 eventual replacement. 

Ear s/n 195 bonded to S3 flat of mass June 29, 2016

Ear s/n 196 bonded to S4 flat of mass June 30, 2016

The first ear (s/n 195) has a series of bubbles along it's edge, but the surface area of these do not add up to more than the allowed amount as per E1000277, so it passes, see pix attached.

The second ear (s/n 196) has no bubbles and is nice full bond. 

C. Cahillane

After taking a close look at the Hanford actuation measurements, with the addition of the January 7th measurements, it became very obvious that the actuation measurements were never compensated for time dependence.
I am not sure if the same is true at Livingston.  I will be investigating shortly.  If so, it could be a major reason why LLO actuation uncertainty is so large... Previously, I used only calibration-week actuation measurements for LHO, meaning the kappas could not have varied too crazily in that period.  But LLO had measurements more spread throughout the run.

This is obviously quite important for our uncertainty budget... The systematic errors will be affected on the order of the kappas that were never applied, so about 3% for TST and 2% for PUM and UIM.  The uncertainties will be smaller since removing time-dependence will cluster our measurements better.

I have posted a plot similar to the one from aLOG 27290 showing the actuation magnitude variances and covariances between each day's measurements.  
Blue is the measurements without the kappas applied.  Red is with the kappas applied.

Comparing to aLOG 27290, we see UIM and PUM variances stay about the same, which makes sense because kappa_pu is not that volatile.
But TST variance goes from 2.5% to 2.0%!

I have added the following scripts to the calibration repository:

analyze_pcal_20150928_kappa_corr
H1PCALparams_20150826_kappa_corr
H1PCALparams_20150828_kappa_corr
H1PCALparams_20150829_kappa_corr
H1PCALparams_20160107_kappa_corr

These allow me to correct the LHO time dependence in our actuation functions.

Jeff, Kiwamu, Stefan

We had trouble engaging the ISS 2nd loop at 40W. The core issue is that the digitized signal (H1:PSL-ISS_SECONDLOOP_SIGNAL_OUTPUT) swings rail to rail without the loop, making it difficult to properly pre-set the offset adjuster (H1:PSL-ISS_SECONDLOOP_REF_SIGNAL_ANA).
As a result, when engaged, the ISS 1st loop tended to swing into saturation.

I patched the Guardian as follows:
- Instead of trying to rely on a calculation for the offset adjuster (H1:PSL-ISS_SECONDLOOP_REF_SIGNAL_ANA), I hard-coded it to 0.987 V, which is where it likes to sit at 40W. (THis is obviously not a long-term solution.)
- As soon as the servo comes on, I ramp H1:PSL-ISS_SECONDLOOP_REF_SIGNAL_SERVO_GAIN to zero, clear the history, and re-engage it on the diffreation power.

This seemed to engage the 2nd loop without excessive transient - back to full locking.

PS: I might have already mentioned this: but the ISS SHOULD BE AC COUPLED!

Code:

In PREPARE_ISS:

        # read current servo signal value. If it is too far from the operating point, correct the offset.
        #second_loop_out = cdu.avg(1, 'PSL-ISS_SECONDLOOP_SIGNAL_OUT16')
        #if abs(second_loop_out) > 1.0:
        #    # measure current input PD value
        #    # Sets the ref. signal value to bring it to close to the operating point
        #    pd_avg = cdu.avg(1, 'PSL-ISS_SECONDLOOP_PD_14_SUM_INMON')
        #    ezca['PSL-ISS_SECONDLOOP_REF_SIGNAL_ANA'] = round(0.5*((pd_avg*2.29*.0006103515625)+0.101), 6)
        #    time.sleep(5)

        # above section commented out - instead command the value that worked  (Stefan Ballmer 20160629 for 40W)
        ezca['PSL-ISS_SECONDLOOP_REF_SIGNAL_ANA']=-0.9826934814453125
 

In CLOSE_ISS, once the loop is closed:

            # the loop is successfully locked, ramp off the loop (Stefan Ballmer 20160629 for 40W)
            ezca['PSL-ISS_SECONDLOOP_REF_SIGNAL_SERVO_GAIN'] = 0

and a little later:

            # clear and ramp on the loop (Stefan Ballmer 20160629 for 40W)
            ezca['PSL-ISS_SECONDLOOP_REF_SIGNAL_SERVO_RSET'] = 2
            time.sleep(0.1)
            ezca['PSL-ISS_SECONDLOOP_REF_SIGNAL_SERVO_GAIN'] = 1
 

While we were centering WFSA and WFSB we discovered that adjusting the beam in pitch shows up as yaw on the WFS medm, and adjusting the beam in yaw shows up as pitch on the medm.

It's unclear how long this has been the situation.

In this morning, I checked the OMC DCPD electronics chain (for both A and B) by injecting known sine wave analog signal. This was one of those items that Keita suggested me a while ago.

According to the data, I am concluding that our calibration model needs to add another pole at 10-ish kHz for accurately simulating the OMC whitening circuits.

Method

The measurement method is straightforward -- it is a swept sine measurement in a manual way.

I had a function generator by the HAM6 electronics rack which drove a single-ended-to-differential convertor (D1000931, technically speaking this is a coil driver test box). Then the differential signal is sent to the input of the OMC DCPD whitening board (D1002559, S1101603) by some clipping technique. By the way, the actual cable for connecting the OMC DCPDs were unplugged during the measurement. The excitation amplitude was set to 2 Vp-p at the function generator which resulted in 2 Vp-p in both positive and negative paths at the input of the whitening board as expected accroding to the schematic of the coil driver test box.

I then recorded the data in IOP at the full sample rate using dataviewer for 1 sec for a selected excitation frequency (and for some reason, diaggui did not want to run and hence dataviewer this time). Keeping the same excitation amplitude, I manually stepped the freqyency from 8 kHz to 100 Hz. After every step, I saved the time series of the IOP so that I can make a transfer function later. In addition, I had an oscilloscope with me which kept monitoring the excitation ampitude at the input of the whitening board. The scope told me that the excitation ampltude stayed constant at 2.02 Vp-p in each channel throughtout the measurement. The OMC DCPD had

Analysis and result

To get a transfer function from the data that I took in time series, I decided to do a sine-wave fitting for each data chunk to get the amplitude information. I wrote a small matlab script to do it. It can be found at:

aligocalibration/trunk/Runs/ER8/H1/Scripts/OMCDCPDs/Matlab/OMC_DCPD_AnalogChain_TimeSeries.m

The script utilizes fminsearch and spits out the best fit amplitude for each frequency measurement. Additionally, it places uncertainty or error bar by taking the standard deviation of the residual. Note that this is not a standard way to place an error bar since it does not take the number of measurement points into consideration. According to the fit, the residuals were found to be usually a few counts which is much smaller than the amplitude of signals which was about 2000 counts. So it typically places 0.1% uncertainty after all.

The result is shown in the attached pdf. By the way the lower panel in the plot says, "residual", but it should read "(measured)/(model)" instead. It shows the measured transfer function together with the expected model transfer function for comparison. It is very obvious that the measurement suggests that our model is missing some high frequency pole. The model is merely made of the analog AA response which I have already measured and fitted. Adding some random pole, I could see an extra pole at around 10.7 kHz making the fitting much better. In fact, I sort of knew that there seemed to be a high frequency pole by some other measurements which I did not post. We probably need to add this high frequency pole in our calibration model.