Kiwamu, Nutsinee
DIAG_MAIN complained about HWSY having bad peak counts so we investigated. The first plot shows both HWS mean sled power and peak counts trend over the past 60 days. *HWSY sled is losing 23 mW per day (running at 103 mA current) while HWSX sled is losing 6.5 mW per day (running at 98 mA current). HWSY SLED was replaced back in April 26th, 2015.
Here I attached screenshots of what the stream images looks like right now for both HWSX (right) and HWSY (left). The bright spots on HWSY stream images are not as bright compared to HWSX, suggesting that the low number of peak counts is caused by the deterioration of the sled power ("peak counts" are refered to number of red spots around the center).
We have some spares (I think Y sled is 840 nm. We have three of those). One thing we could do here is to replace the sled next Tuesday so we can get some useful (live) data during ER9.
Commissioners are using TCS HWS to investigate any unexpected absorbtion of the ITM optics. It can be use to look at the affect of the CO2 heating real time. So it's important to keep this thing running.
I keep the HWSY code running for now since Aidan was able to use the recorded data for offline analysis.
----------
*Later I found out that the power calibration now is slightly off from calibration back in March 1st (alog25806). The below I attached the calibration as of today (Been these values since March 23rd). HWSY calibration stays the same but the rate of power loss per day for HWSX would be overestimated by ~30%. This also affects the previous analysis of sled power deterioration in alog26341 slightly.
Aidan has confirmed that HWSY code output bad data on his end. I stopped the code so it won't write anymore junk data to the disk.
I looked at 1 hour of the input power / IMC power oscillations that occurred last weekend and how that power change effected the alignment of the IO IMs, IM1, IM2, IM3, and IM4.
4 attachments:
One explanation for the difference of IM2 compared to IM1, IM3, and IM4, is that the beam on IM2 is no longer centered, so the change in IMC power causes the optic to change alignment.
If this is true, the change of the beam centering on IM2 could be big enough to have significantly changed the alignment of the beam through the Input FI.
UPDATE: IM1 glitching
IM1 UR OSEM signal is glitching and the glitches can be seen in DAMP_P and DAMP_Y at about 0.1urad in both signals.
These glitches do not show up in IM4 Trans.
UPDATE: IM2 oscillations clearly seen on IM4 Trans:
amplitude (p-p) | |
IM4 trans pitch | 0.035 |
IM2 pitch | 0.25urad |
IM4 trans yaw | 0.015 |
IM2 yaw | 0.25urad |
Plot attached shows IM4 trans pitch and IM2 damp_p in blue, IM4 trans yaw and IM2 damp_y in green.
On Tuesday afternoon, July 5, I plan to update the code on all of the Beckhoff vacuum controls computers (h0vaclx, h0vacly, h0vacmr, h0vacmx, h0vacmy, h0vacex, h0vacey). This is to use the new PID controller that has been installed and tested at end Y and mid X. This will trip the ESD high voltage at end X and end Y. Also the cold cathode gauges will turn off and may take some time to refire. This falls under WP 5972.
J. Kissel, T. Sadecki, E. Merilh We're find that all of the QUADs are saturating much more than usual this morning. Tracing the large control signals, it looks like that HARD loops' error signals are much larger than they were during the -12 hours ago lock stretch, and have been increasing since half way through. The investigation is still preliminary, but we've lost two locks already to this excess noise. Note that this is all well before any of the ISC_LOCK guardian's LOWNOISE_ASC step of which Jenne told us to be mindful (see LHO aLOG 28109). I have a suspicion it's something obvious to the ISC experts, but this is what the engineering team gets for trying to replicate such rapid improvement from late last night! The investigation continues... Saturations start happening as soon as the CHARD Loops come on in ENGAGE_SRC_ASC.
Added 1296 channels. Removed 589 channels. (changes attached)
yesterday fw0 went very unstable for a few hours. We noticed that during this period file access on the QFS server (h1ldasgw0) was normal, but even simple file actions like a long listing (ls -al) from h1fw0 could take many seconds. Rebooting and power cycling both machines did not fix this. We configured h1fw0 to only write the science frame, which did make it more stable. At this point the CRC epics channels for fw0 and fw1 were different, but the two science frames written were byte identical. It looks like the CRC is the checksum of the data block the frame writer needs, if it is not writing the commissioning frame then it is the CRC of the science data only. At this point we also did full DAQ restart (17:37PDT) to resync everything. At this point we felt that having the CRCs match is an important diagnostic, so h1fw0 was reverted to write both frames again, and this time it went stable.
h1fw0 has been stable since the 17:37PDT DAQ restart (we don't know why). h1fw1 has restarted 3 times since then (01:04, 01:32, 07:03 PDT Friday 7/1). I think this level of fw1 instability is acceptible if fw0 continues to be 100% stable.
Our investigation suggests that Keith's suggestion of upgrading the NFS link between FW and QFS-NFS server from 1GE to 10GE would be a good test. Dan has some fiber-optics 10GE cards we could borrow. We will schedule this test after ER9 or before if the instability reappears.
J. Kissel Since Jenne was able to get a beautifully long 6+ hour lock stretch last night (see LHO aLOG 28109), I've moved the PCALX high frequency calibration line frequency up to 2501.3 Hz. I've also increased the amplitude of the line to 40 000 [ct] of excitation in H1:CAL-PCALX_PCALOSC1_OSC_SINGAIN. I'm not *sure* it matters (since Kiwamu just recently updated the OMC DCPD whitening filter compensation; see LHO aLOG 28087), but let's be mindful that Jenne had added a stage of OMC whitening during that long lock stretch. It looks like she turned on the whitening right when the inspiral range jumps from just under 40 to just under 50 Mpc, so we should just consider the whole lock stretch having the first stage of whitening ON. I've accepted the new frequency (and increased amplitude) in SDF under the OBSERVE.snap file.
Jenne, Carl, Ross
We have been testing some of the filters settings for PI mode damping. During the lock tonight we saw 5 unstable modes at 15009Hz (ETMY), 15522Hz (ITMX), 15541Hz (ETMX) , 15542 (ETMY) and 18038Hz (ETMY). The 15522, 15541 and 15542 modes are all unstable after four hours of the 40 W lock. The other two are unstable temporarily as the thermal transient passes through them.
We have demonstrated that we can get steady state damping of the three unstable modes using either 10 Hz band pass filters or iwave tracking (first plot shows damped state amplitudes). We will leave the it in the 10 hz band pass filter configuration. We also show that we can excite and damp the steady state mode at 15009Hz using a 10 Hz bandpass filter (second plot) which should result in a reduced mode amplitude during the transient.
We will put these filter settings into the guardian. The ETMX damping will not happen unless the SUS_PI guardian state is manually set to ETMX PI DAMPING.
Between 11 to 12 UTC all four ring heaters were stepped by 0.5W for approximately 15 minutes for test mass mode identification.
I lowered the gain of the ASC HARD loops pretty significantly, and that has drastically improved our DARM sensitivity.
It's starting to be hard to see where the coherence is coming from, but it looks like it's not really any of the ASC loops above 10 Hz. (The coherences plots in the attachment were taken while A2L was running, so the 8 lines around 20Hz are ignore-able.)
I ran A2L again, and there weren't any big changes, or improvements to the sensitivity.
I turned off the MICH and SRCL feedforward for a while, so that I could see about re-tuning the FF filters. I did SRCL injections starting at 09:00 UTC, MICH injections starting at 9:14 UTC, and PRCL injections starting at 9:23 UTC.
Attached is a screenshot of my current HARD loop settings. I have put them into guardian in the new LOWNOISE_ASC state that happens just before we switch to the lownoise ETMY ESD, but have not run the new code for this state yet. Be mindful of this when going to lownoise tomorrow.
Other than PI damping work ongoing, the IFO is undisturbed, starting at 9:40 UTC.
Oh, and I know that we decided to run ER9 without OMC DCPD whitening in case the PIs got too big, but since they're okay right now, I have the whitening on so we can see the pretty new shot noise level that we have.
TITLE: 07/01 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Lock Aquisition
INCOMING OPERATOR: None
SHIFT SUMMARY: Eventually got a solid 2.5hr lock in "low noise" and Jeff was able to finish up his measurements. Before that we had two lock losses caused by bounce modes very quickly runging up, although one was exactly 9.80Hz which falls between our other known bounce modes. PI was the cause of the long lock, ITMX. Locking has been the same as it has the past few days (AS I typed that we started having issues getting through turning on the ASC).
LOG:
J. Kissel, J. Driggers, T. Shaffer, D. Tuyenbayev, E. Goetz, K. Izumi Over two ~1-2 hour lock stretches, I've managed to get the measurements needed for a baseline calibration update for ER9. Complete success! We'll work on the data analysis tomorrow, but I list the locations where all measurements have been committed to the CAL repo below. Of particular notes for the configuration of the IFO while doing these measurements: - The OMC DCPDs have *no* stages of whitening employed. We've decreed that given the unknown success rate of PI damping over the next few days, it's more robust to leave the whitening off. We're not gaining too much in the high frequency end of the sensitivity anyways. - All suspensions, including ETMY, have had their PUM stage switched to "Acq ON, LP OFF," i.e. state 2, or the highest range (i.e. not low noise). After some digging, Jenne found this was changed for some reason about a month ago, and maybe a setting that got lost in the power outage or something. I don't think either of these seemingly detrimental configurations are all that bad for ER9, given the bigger sensitivity issues elsewere. Also note, in order to break the correlations we've found in O1 data between measurements of Actuation Stage Strength (see e.g. LHO aLOG 28096), I've taken an independent PCAL2DARM sweep for every isolation stage. Measurements needed for Sensing Function: /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/PreER9/H1/Measurements/PCAL/ 2016-07-01_H1_PCAL2DARMTF_4to1200Hz_SRCTuned.xml Exported as: 2016-07-01_PCALY2DARMTF_4to1200Hz_A_PCALRX_B_DARMIN1_coh.txt 2016-07-01_PCALY2DARMTF_4to1200Hz_A_PCALRX_B_DARMIN1_tf.txt /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/PreER9/H1/Measurements/DARMOLGTFs/ 2016-07-01_H1_DARM_OLGTF_4to1200Hz_SRCTuned.xml 2016-07-01_H1_DARM_OLGTF_4to1200Hz_A_ETMYL3LOCKIN2_B_ETMYL3LOCKIN1_tf.txt 2016-07-01_H1_DARM_OLGTF_4to1200Hz_A_ETMYL3LOCKIN2_B_ETMYL3LOCKIN1_coh.txt 2016-07-01_H1_DARM_OLGTF_4to1200Hz_A_ETMYL3LOCKIN2_B_ETMYL3LOCKEXC_tf.txt 2016-07-01_H1_DARM_OLGTF_4to1200Hz_A_ETMYL3LOCKIN2_B_ETMYL3LOCKEXC_coh.txt Measurements needed for Actuation Function: /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/PreER9/H1/Measurements/FullIFOActuatorTFs/ 2016-07-01_PCALYtoDARM_FullLock_L1.xml 2016-07-01_H1SUSETMY_PCALYtoDARM_ForL1Drive_FullLock_tf.txt 2016-07-01_H1SUSETMY_PCALYtoDARM_ForL1Drive_FullLock_coh.txt 2016-07-01_H1SUSETMY_L1toDARM_FullLock.xml 2016-07-01_H1SUSETMY_L1toDARM_State1_FullLock_tf.txt 2016-07-01_H1SUSETMY_L1toDARM_State1_FullLock_coh.txt 2016-07-01_PCALYtoDARM_FullLock_L2.xml 2016-07-01_H1SUSETMY_PCALYtoDARM_ForL2Drive_FullLock_tf.txt 2016-07-01_H1SUSETMY_PCALYtoDARM_ForL2Drive_FullLock_coh.txt 2016-07-01_H1SUSETMY_L2toDARM_FullLock.xml 2016-07-01_H1SUSETMY_L2toDARM_State2_FullLock_tf.txt 2016-07-01_H1SUSETMY_L2toDARM_State2_FullLock_coh.txt 2016-07-01_PCALYtoDARM_FullLock_L3.xml 2016-07-01_H1SUSETMY_PCALYtoDARM_ForL3Drive_FullLock_tf.txt 2016-07-01_H1SUSETMY_PCALYtoDARM_ForL3Drive_FullLock_coh.txt 2016-07-01_H1SUSETMY_L3toDARM_LVLN_LPON_FullLock.xml 2016-07-01_H1SUSETMY_L3toDARM_LVLN_LPON_FullLock_tf.txt 2016-07-01_H1SUSETMY_L3toDARM_LVLN_LPON_FullLock_coh.txt
J. Kissel, E. Goetz Just to post a status report before I disappear for the 4th, Evan and I have used Evan's new infrastructure to produce a model of the DARM open loop gain and sensing function from the above measurement. As one can see, there's still some work to do cleaning up the systematics in the model, but we're close. There're several things that are immediately evident: - We have not changed the model's optical gain value from O1, so it's not terribly surprising that the optical gain model is high by 20%. - There is total and obvious detuning. So much so, that I think this is what's causing the severe drop in open loop gain. - The drop in open loop gain is pretty nasty -- it causes sharp gain peaking at 10 Hz (as shown by the screenshot of the DTT template). Kiwamu's working on a fitting routine that is similar to Evan's MCMC results from O1 (see T1500553), but advancing those results to include the effects of detuning so that we can add this to the model. Stay tuned! Lots of work to do before Wednesday!
Evan G., Jeff K.
I processed the suspension actuation coefficients for the L1, L2, and L3 stages using the preliminary DARM model based on the one used during O1. We know that there need to be some modifications made, but the take home message here is that the actuation coefficients are about as to be expected. All are within ~5% of their O1 values. See attached figures.
The first attachment shows the UIM stage actuation coefficient. We have not yet included the BOSEM inductance and we have not yet included any actuator dynamics, so there is remaining discrepancy above ~20 Hz. There also appears to be some sort of phase wrapping issue that we still need to sort out.
The second attachment shows the PUM stage actuation coefficient. Things look pretty good here, although there may be some fluctuating optical gain which we have not yet accounted for in the measurement.
The third attachment shows the TST stage actuation coefficient. Again, there may be some fluctuating optical gain not accounted for in the measurement, and there is a phase wrapping issue to be sorted out.
Next on the agenda is to sort out the above issues and establish the actuation coefficients for the ER9 run.
The sensing function attached above includes uncompensated high frequency poles from the whitening chassis and the transimpedence amplifier, so there will be some delay assocated with these values.
To see only the optical response, we have removed these high frequency poles, as well as the analog AA and digital AA transfer functions, in the attached data file.
J. Kissel After a nice long ~1 hr lock stretch at 41.5 W, I've moved the long-duration PCALX "sweep" from 1501.3 to 2001.3. I've accepted the new frequency on PCALX's SDF system (under the OBSERVE.snap file). We'll need a lock stretch of at least 2hrs for this line. Recall we're following the same plan as was done in post O1; see e.g. LHO aLOG 24843.
No clue for the mystery noise, but it's not the jitter from HPL linearly coupling to DARM.
In the attached, you see coherence between darm and usual things as well as DBB beam jitter signals measuring the HPL jitter. Nothing is obvious about mystery noise at e.g. 100Hz.
DBB PMC was locked, aligned using WFS, and WFS control was turned off. IFO was locked at nominal low noise with 40W.
We will gain something by retuning SRCL and MICH FF for f<100Hz, but these are not the main sources of the mystery noise at e.g. 100Hz. We might also benefit from pushing the CM gain to lower high kHz noise.
There's some coherence with ISS second loop sensors between 100 and 1000Hz. (Note that these ISS second loop signals are NOT what is plugged into the second loop board, they are digitally generated from indivisual PD output and that's how we still retain PD58 sum.) It's not clear if this is the intensity noise, changing ISS second loop gain might reveal something but this cannot be the mystery noise.
DBB_QPD_1DX etc. are WFS errors, and DBB_QPD_1QX etc. are WFS DC pointing. Apparently there's some jitter peak at 1kHz which shows up a bit in DARM, and there might be something at about 30-40Hz, but except these, nothing. PIT WFS signals (PSL-DBB_QPD_1DY and 2DY, these are blue and brown in the middle panel) might be showing the same broad 100-1000Hz thing that the ISS sees.
In the second attachment, I measured the IMC WFS and it shows some structures which we might have to do something about once we get rid of the mystery noise, but it doesn't look as if linear coupling of the jitter explains the mystery noise. (In this second attachment, strange bump in DARM at 300Hz is because something was injected for calibration measurement.)
[Marie, Jenne]
Marie suggested that we run A2L, which we did. She was totally right - we should have done this a week ago! We win about an order of magnitude in sensitivity at low frequencies. This will make things much easier for the calibration team.
Attached is a spectrum showing the enormous improvement. This reduced the coherence between DARM and CHARDY, but not to the low level that CHARDP is, so there is probably more that we could do to get even better sensitivity with A2L.
Edit: The new A2L values have been saved in both the down.snap and observe.snap sdf files.
I have measured the transfer functions of the newly installed whitening chassis (alog 28010, ECR:E1600192, SN:S1101627) for the OMC DCPDs. I used LISO for fitting the data.
[1st stage for DCPD A]
Best parameter estimates:
pole0:f = 18.6776675365k +- 10.45 (0.0559%)
pole1:f = 14.3409807014k +- 6.292 (0.0439%)
pole2:f = 98.9444053052k +- 32.86 (0.0332%)
pole3:f = 10.3243596197 +- 359.4u (0.00348%)
zero0:f = 985.3423393170m +- 79.51u (0.00807%)
factor = 998.8350760502m +- 67.01u (0.00671%)
Final chi^2=0.00252952
[2nd stage for DCPD B]
Best parameter estimates:
pole0:f = 18.5698573085k +- 14.71 (0.0792%)
pole1:f = 14.6043378758k +- 9.31 (0.0637%)
pole2:f = 100.4496012696k +- 43.14 (0.0429%) > MAX
pole3:f = 10.0731025960 +- 470.5u (0.00467%)
zero0:f = 961.4535554217m +- 104.7u (0.0109%)
factor = 997.7425128879m +- 90.38u (0.00906%)
Final chi^2=0.00454374
[Small notes]
I have also measured the same transfer functions with a 4 times smaller excitation signal in order to check some sort of saturation-induced measurement error. I did not quantitatively look at these data, but plotting them on top of the above data showed excellent agreement (to my eyes).
[SVN info]
All the data, scripts and figures are checked into SVN at:
/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O2/H1/Scripts/OMCDCPDs
/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O2/H1/Measurements/OMCDCPDs/2016-06-30
/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O2/H1/Results/OMCDCPDs
Between yesterday and today, Gerardo and I successfully bonded the ears to core optic mass ETM15, destined for LLO for post-O1 eventual replacement.
Ear s/n 195 bonded to S3 flat of mass June 29, 2016
Ear s/n 196 bonded to S4 flat of mass June 30, 2016
The first ear (s/n 195) has a series of bubbles along it's edge, but the surface area of these do not add up to more than the allowed amount as per E1000277, so it passes, see pix attached.
The second ear (s/n 196) has no bubbles and is nice full bond.
Inspection of the bonds this morning showed no change to the nice S4/Ear surface bond.
The small edge bubbles along the S3 Ear have migrated a bit (not worse, just moved a bit). Therefore, we decided to rewet the edge with solution to see if it would wick in and close some of the bubbles. This worked to close one of the larger bubbles, but one bubble continued to morph so we'll reinspect Monday. Note, none of the sizes of the bubbles got bigger, so the surface area of the bubble total remains the same and below spec.
Attached is a picture of what the bubbles in the ETM15 S3 Ear bond look like today (recall, it was bonded last Wed so it has now been over 5 days since initial bond). Apparently over the weekend, the bubbles ran inbound a bit further. I estimate the surface are of the bubbles to be near 20mm sq., still under the 50mm sq. spec. E1000278, although it isn't pretty. I measured the longest streaks to be almost 3mm long, with a width of 0.4mm. There are no bubbles inward from the edge near the center - anything shown in the picture is a camera reflection.
The S4 Ear bond looks unchanged from inspection last week, very good bond, very little bubbles.
Pictures of this ear bond taken by Danny at LLO just before use at LLO, indicate that the bubbles have not changed shape or size since the picture attached July 2016. Phew, said both Gerardo and I.
biggest = prm pitch changing by 2.5urad
close second = im4 pitch changing by 1,15urad
plot and chart of top 16 optics/DOFs attached
plot of optics/DOFs in the chart - meant to attach yesterday. I
used OpLev signals when available and OSEMs when not.
UPDATE to my alog's less than descriptive title: I think the original title suggests these changes were during a 40W lock, but this is not the case.
The optic alignment changes are actually happening over 8 minutes of full lock while the input power increased from 22W, the O1 power level, to 40W.
Kiwamu, Stefan
We were trying to nail down what is responsible for the Power Recycling Gain change during power-up, so we went through the whole sequence slowly, and gave the system time to settle in each state:
All times UTC of 20160629:
- 18:54:37 / Green dot: Power-up starts from 2Watt to 10Watt
- 19:00:32 / Red dot: Power-up restarts from 10Watt to 40Watt
- 19:06:10 / Black dot: : Power-up to 40W finishes, but additional SOFT offsets remain left off.
- 19:10:15 / Blue dot: Soft offsets in yaw start ramping on over 2 minutes
- 19:12:17: Soft offsets ramp in yaw is done
- 19:15:17: Soft offsets in pit start ramping on over 3 minutes
- 19:18:19: Soft offsets ramp in yaw is done
- 19:21:20: End of undisturbed period - guardian continued
In particular, we now there is a significan power recycling gain improvement with a soft yaw adjustment of the x-arm. Additionally, the beam splitter cannot affect the relative alignment of x-arm and power recycling cavity. SO attached are only yaw signals from x-arm and power recycling cavity.
Conclusions:
- During the very first step in power up (green dot) there is a momentary shift in PRM yaw and IM4 yaw, but it does not continue during the additional power increase.
- During the main power increase (between red dot and black dot). There is a consiostent signature in the ETMX control, oplev and IR camera signal, sggesting that we are slightly moving the x-arm with the ASC system. THe ITMX doesn't see much of this in control signal and oplev signal (its camera is broken and not plotted). However no significant motion is visible in any of the power recycling cavity mirrors (this includes PR2, which is not plotted).
- Once we start moving the soft loops with soft offsets, ETMX, ITMX, PR3 and BS clearly respond. Interestingly, the ETMX moves to the same direction as during the power increase.
Attached are 220days of recycling cavity alignment data (Nov 15 2014 to now)
The alignment offsets for IM1-3 are not a valid way to track the optic alignment, since IM1, IM2, and IM3 all have significant alignment shifts after a HAM2 ISI trip.
I wrote a procedure to correct these after restoring the ISI and the optics but adjusting the alignment offsets to drive the optics back to their nominal alignments.
During O1 the procedure was to drive the optic back to it's most recent good value, however this can be hard to decipher, so in May I posted a Nominal Alignment for the IMs and have maintained it since.
I've been maintaining an alignment of the IMs based on our last good high power lock in April 2016, and this alignment choice is explained in alog 26916, and recently updated in alog 28016.
On a longer time scale, the alignment of the IMs is harder to track since I only started restoring their alignments after an ISI trip around September 2015, so before that time when IM2 pitch shifted 40urad we spent time realigning H1 to follow that change.
In my investigation, I see that the IM alignment shifts have caused down time in H1, and IM1-3 now have an alert in guardian to prevent the IM alignment shifts from causing down time in O2.
Here are some pertinent alogs from my investigation:
While we were centering WFSA and WFSB we discovered that adjusting the beam in pitch shows up as yaw on the WFS medm, and adjusting the beam in yaw shows up as pitch on the medm.
It's unclear how long this has been the situation.
Keita cleared this up in his alog 27807. The choice of pitch and yaw on the table for WFS matches ptich and yaw in the chamber, so on the table the pitch (yaw) adjustment on the steering mirror to the WFS corresponds to the yaw (pitch) DOF in chamber.