edmond.merilh@LIGO.ORG - posted 09:37, Monday 10 August 2015 (20373)
PSL DBB scans + ISS Scan
Non-image files attached to this report
Attached are plots of the NPRO output power, the output power of the NPRO pump diodes, and the output of the front end laser for the past 4 years. Whilst there's not much wiggle room left on the NPRO pump diodes, there's room left on the power amplifier pump diodes. The laser output should be good through to the end of O1.
The IFO was not locking for hours so I took the SEI to READY at EndX and restarted the Pump Servo. This required a trip to the end station. The controller started without issue and while the platforms were down, I captured safe.snaps for the ISI and HPI. It has been a long time since we've had a pump controller hang like this, maybe I can pull this from the log.
The ISI did trip on the T240s but I don't know exactly what the platform was up to at the time. I isolated fully second attempt no problem.
This is an alog about the (aborted) attempt between Saturday and Sunday to make the interferometer work at 0.43 W of TCS power.
The locking sequence was fine up through the dc readout step. However, the interferometer would lose lock at 10 W or higher. We've seen this kind of behavior before, and the culprit is usually the SRC1 yaw loop. I rephased ASA36 by driving SRC1 yaw at 0.3 Hz. Segements 3 and 4 (the two that we're using) both gained 25°. Segments 1 and 2 still have less signal than 3 and 4, which is what we saw before. At the end of the night I reverted these phasings. I also redid the dark offsets on these segments, and have left them there.
Rephasing ASA36 didn't seem to fix the problem, so I went through and rephased Refl A 9, Refl B 9, AS A 45, Refl A 45, and Refl B 45. For Refl 9, I drove cHard at 8 Hz in pitch . For Refl 45, I drove PR3 at 9.1 Hz. For AS A 45, I drove dHard pitch at 8 Hz. Many of the original segment settings were so far off in phase that I suspect they are bad even at the lower TCS power, so I did not revert these new settings at the end of the night.
I also looked into the subtraction of the Refl 9 signal from the PRC2 loop by driving cHard in pitch again. The ratio of the signal in the 45 MHz and 9 MHz WFS was Refl A 45 / Refl A 9 = −0.79 and Refl B 45 / Refl B 9 = −0.76. Our current subtraction settings in the input matrix assume a ratio of −0.60 for both A and B. We should recheck this at the lower TCS power and update the Guardian settings.
Finally, I looked at rephasing the LSC in full lock. I drove PRCL at 136 Hz and then rotated POP9 from 90° to 95° to minimize the apperance of the line in POP9Q. I repeated this for SRCL, but no rephasing was required. However, the subtraction of POP9I into SRCL was quite bad. I drove PRCL again while watching the SRCL error signal, and based on this I changed the matrix element for POP9I→SRCL from −0.04 to −0.012. This setting is in the Guardian (and should be rechecked at the lower power).
The CW injections were turned back on at gps = 1123224216. I believe they have been off since Jeff noticed on 21 July: https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=19817 The hwinj team is in the process of setting up a monit / crontab to auto-restart the CW injections after failures.
J. Kissel, D. Tuyenbayev. At the request of Duncan MacLeod, we've svn updated the CAL_INJ_CONTROL.adl MEDM overview screen for the hardware injection system.
J. Kissel, C. Cahillane, J. Driggers Today we explored some of the potential systematics in using the ALS DIFF VCO to calibrate the DARM ETM actuation scale factor, as has been done prior to ER7 (see LHO aLOG 18711). Specifically, we've tried to precisely measure the frequency dependence of the the famous z:p = (40:1.6) [Hz] filter just before the low-noise VCO (see LowNoiseVco, and D0900609). However, because the VCO (an MFC9119-10) is a non-linear actuator, it must be locked to a carrier frequency. We want the carrier to be what is nominally used to control the arms during ALS DIFF using the same Phase-Locking Loop (D1300812) and Frequency Difference Divider (see T1400317) to remain in the regime where we get the same amount of [Hz/ct] out of the PLL's control signal. I attach a PDF of the entire system which is a copy of what was discussed in T1500383. Also attached are plots of the measurement. The idea: - Lock the PLL to an independent frequency reference (a marconi in the CER) in place of the DIFF RFPD input to the Phase-Frequency discriminator. The IFR carrier is tuned to be of frequency roughly the nominal ALS DIFF frequency 157.84 [MHz], but refined further to make the H1:ALS-C_DIFF_PLL_CTRL_INMON as close to zero as possible. It's that PLL_CTRL signal that's used as an error signal for the ALS DIFF arm feedback when ALS DIFF is locked. Today, the carrier was tuned to 157.783 [MHz] to minimize PLL_CTRL. Again, this ensures that the [Hz/ct] of the DIFF_CTRL_INMON signal (or more precisely the VCO itself) remains in the linear regime exactly surrounding the carrier frequency we use for during a nominal ALS DIFF lock. - Measure the PLL Open Loop Gain TF, using an SR785 connected to the PLL (whose inputs are in the back of the rack); Source -> EXC, Test2 -> Ch1A (Ch1B 50 [Ohm] terminated), Test1 -> Ch2A (Ch2B 50 [Ohm] terminated). Once we have the Open Loop Gain transfer function -- down to the very *very* low frequency of ~1 [Hz], then we can model the entire loop, extracting out all of the frequency dependence -- namely, the 40:1.6 Hz filter. It turns out, measuring this OLGTF is hard and time consuming, given the extreme amount of gain the loop has at low frequency. In other words, in the nominal configuration, with a 55 [kHz] UGF loop and boosts, the PLL is *very* good at suppressing your 10 [Vpp], ~1-100 [Hz] excitation, so you have to drive at the limit of your analog driver, and integrate for a very long time. Indeed, the only way to get *any* coherence below ~1 [kHz] is to turn the boosts OFF. As such, we took a ~2 [hr] long TF of the loop with boosts OFF, and then characterized the boosts independently. Craig and I continue to model the results, but the GPIB templates for the measurements, the measurements themselves, and notes on the measurements live here: /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/PreER8/H1/Measurements/ALSDIFF/2015-08-09/ # notes Notes_ALSdiffPLL_9Aug2015.txt # templates TFSR785_DiffBoostsOLGTF_10_100k_09Aug2015.yml TFSR785_DiffPLLOLGTF_0p5_200_09Aug2015.yml TFSR785_DiffPLLOLGTF_100_100k_09Aug2015.yml # olgtfs PLLOLGTF/TFSR785_09-08-2015_144001.txt << High Freq, Boosted PLLOLGTF/TFSR785_09-08-2015_144447.txt << High Freq, No Boosts PLLOLGTF/TFSR785_09-08-2015_155915.txt << Low Freq, No Boosts # boost TFs PLLBoostTF/TFSR785_09-08-2015_190215.txt << Boost 1, z:p = (2k:40) [Hz] PLLBoostTF/TFSR785_09-08-2015_185858.txt << Boost 2, z:p = (17k:2k) [Hz] As it stands, the UGF of the ALS DIFF PLL (in the nominal, boosted configuration) is 54.8 [kHz], with a phase margin of 48 [deg]. Many thanks to Jenne for her help today!
Jenne, Gabriele, Sheila
After the calibration work and before the Solomon Islands earthquake, we got a chance to try phasing refl 9 WFS. The first of the attached screenshots show the phase before we started, and the second shows the phases now.
We tuned them by injecting a line in the mode cleaner length at 11Hz, with the ASC on and 3 Watts of input power. When we started the signal in I and Q was about equal, when we finished most of the quadrants had a factor of 3-4 more signal in I than Q. REFL B quadrant 1 was strange, we weren't able to change the amplitude of the line, but the noise in the two quadrants was clearly changing as we rotated the phase.
We then made a pitch excitation in CHARD and saw about 8 dB more signal in I than Q for both REFL A and B.
We think these settings should be OK for relocking, because the IFO stayed locked while we were changing them. We haven't tested this though because of the earthquake.
J. Kissel, J. Betzweiser, D. Barker WP: 5418 ECR: E1500332 II: 1091 I'll put in a more detailed entry describing all the changes once I've finished with the MEDM screen updates, but I've installed an updated CAL-CS front-end model as per all of the documentation mentioned above. This required a DAQ restart, which thankfully was much more successful than is has been in the recent past. I've confirmed that all old CAL model, bare-essentials, functionality is up and running: - Whitening filter recently installed for DELTAL_CTRL has been moved to DELTAL_CTRL_SUM, because the filter bank has changed name - Turned on the newly named DARM_LINE oscillator at 37.3 [Hz], with an amplitude of 0.08 [ct] (I don't think this amplitude is right; I'll confirm the amplitude with Darkhan shortly, it hasn't been aLOGged). - Filled in the new TM_OUTPUT_MTRX to pass through ETMY into DELTAL CTRL. I can see that this functionality has returned, as Evan is able to get the IFO back to some power level, and the DELTAL ASD lies roughly on the reference. Stay tuned for more detail!
You can follow along in T1500377. Or said more correctly, you'll *need* this document open to help follow along. We have found actuation strength of the QUAD is varying slowly as a function of time, and we're quite confident that slow, time-dependent, charge on the test mass is changing the test mass actuation strength alone (relative to the upper stages of the quad). As such, and before the PUM / TST crossover is in the gravitational wave band, we need to be able to independently track the ESD strength as a function of time, with respect to the overall actuation strength. We have explored several different methods for potentially compensating the variation that we see (see T1500422 for discussion), and the method that results in the least amount of systematic error in the correction needs new front-end infrastructure. This update is to install said infrastructure. Some of the details: (1) New channels stored into science frames for storing each stage of actuation independently. These new channels are stored at 4096 [Hz]. $(IFO):CAL-DELTAL_CTRL_TST_DQ $(IFO):CAL-DELTAL_CTRL_PUM_DQ $(IFO):CAL-DELTAL_CTRL_UIM_DQ Just like the $(IFO):CAL-DELTAL_CTRL_DQ channel, these channels will be whitened to avoid single-precision errors. Note that, for now, I've installed the same whitening filters, i.e. zpk([1;1;1],[500;500;500],1,"n) but we'll need to do the same kind of study that Shivaraj did to make sure they're not limited by precision errors. Also, in doing so, I've *reduced* the frame rate for $(IFO):CAL-DELTAL_CTRL_DQ also to 4096 [Hz]. Note, for offline analysis, this means one now needs to account for the DAQ down sampling filter between 16 [kHz] and 4 [kHz], which is detailed as the "x4" filter in T1400719. (2) There are LOTS of new EPICs records to support the calculations defined in Eqs. 9, 11, 12, and 15 of T1500377. While we thought that the GDS calibration pipeline would be able to simply absorb EPICs records that housed the model parameter values (e.g. A_{0}^{tst}, A_{0}^{pu}, C_{0}, G_{0}) at each calibration line frequency, Maddie tested the new code, and found memory issues with all of the calucaltions that were necessary. As such, now, we not only store the "raw" parameters values, but from those values, the contant / reference / "_{0}" terms in the above mentioned equations are calculated in the front-end. It's a mess of complex analysis, but the new EPICs records, and there associated relation to the above equations are mentioned below. for f_{tst} @ ~ 35 [Hz] User entered EPICs records: $(IFO):CAL-CS_TDEP_ESD_LINE1_REF_OLG_$(REAL,IMAG) G_{0} @ f_{tst} $(IFO):CAL-CS_TDEP_ESD_LINE1_REF_D_$(REAL,IMAG) D @ f_{tst} $(IFO):CAL-CS_TDEP_ESD_LINE1_REF_C_$(REAL,IMAG) C_{0} @ f_{tst} $(IFO):CAL-CS_TDEP_ESD_LINE1_REF_C_NOCAVPOLE_$(REAL,IMAG) C_{res} @ f_{tst} $(IFO):CAL-CS_TDEP_ESD_LINE1_REF_A_TST_$(REAL,IMAG) A_{0}^{tst} @ f_{tst} $(IFO):CAL-CS_TDEP_ESD_LINE1_REF_A_PUM_$(REAL,IMAG) A_{0}^{pum} @ f_{tst} $(IFO):CAL-CS_TDEP_ESD_LINE1_REF_A_UIM_$(REAL,IMAG) A_{0}^{uim} @ f_{tst} $(IFO):CAL-CS_TDEP_ESD_LINE1_REF_A_TOP_$(REAL,IMAG) A_{0}^{top} @ f_{tst} Calcalulated records: $(IFO):CAL-CS_TDEP_ESD_LINE1_REF_A_TST_MODSQ |A_{0}^{tst}|^2 @ f_{tst} $(IFO):CAL-CS_TDEP_ESD_LINE1_REF_A_TST_NEGINV_$(REAL,IMAG) - (1 / A_{0}^{tst}) @ f_{tst} for Eq. 9 $(IFO):CAL-CS_TDEP_ESD_LINE1_REF_A_USUM_$(REAL,IMAG) A_{0}^{pu} @ f_{tst} = A_{0}^{top} + A_{0}^{uim} + A_{0}^{pum} $(IFO):CAL-CS_TDEP_ESD_LINE1_REF_A_USUM_MODSQ |A_{0}^{pu}|^2 @ f_{tst} $(IFO):CAL-CS_TDEP_ESD_LINE1_REF_A_USUM_INV_$(REAL,IMAG) (1 / A_{0}^{pu}) @ f_{tst} $(IFO):CAL-CS_TDEP_ESD_LINE1_REF_C_MODSQ |C_{0}|^2 @ f_{tst} $(IFO):CAL-CS_TDEP_ESD_LINE1_REF_INVCLG_$(REAL,IMAG) (1 + G_{0}) / C_{0} @ f_{tst} for Eq. 9 for f_{pcal} @ ~ 35 [Hz] (which are currently running on PCALY) User entered EPICs records: $(IFO):CAL-CS_TDEP_PCALY_LINE1_REF_OLG_$(REAL,IMAG) G_{0} @ f_{pcal} $(IFO):CAL-CS_TDEP_PCALY_LINE1_REF_D_$(REAL,IMAG) D @ f_{pcal} $(IFO):CAL-CS_TDEP_PCALY_LINE1_REF_C_$(REAL,IMAG) C_{0} @ f_{pcal} $(IFO):CAL-CS_TDEP_PCALY_LINE1_REF_C_NOCAVPOLE_$(REAL,IMAG) C_{res} @ f_{pcal} $(IFO):CAL-CS_TDEP_PCALY_LINE1_REF_A_TST_$(REAL,IMAG) A_{0}^{tst} @ f_{pcal} $(IFO):CAL-CS_TDEP_PCALY_LINE1_REF_A_PUM_$(REAL,IMAG) A_{0}^{pum} @ f_{pcal} $(IFO):CAL-CS_TDEP_PCALY_LINE1_REF_A_UIM_$(REAL,IMAG) A_{0}^{uim} @ f_{pcal} $(IFO):CAL-CS_TDEP_PCALY_LINE1_REF_A_TOP_$(REAL,IMAG) A_{0}^{top} @ f_{pcal} Calcalulated records: $(IFO):CAL-CS_TDEP_PCALY_LINE1_REF_A_USUM_$(REAL,IMAG) A_{0}^{pu} @ f_{pcal} = A_{0}^{top} + A_{0}^{uim} + A_{0}^{pum} $(IFO):CAL-CS_TDEP_PCALY_LINE1_REF_G_MODSQ |G_{0}|^{2} @ f_{pcal} $(IFO):CAL-CS_TDEP_PCALY_LINE1_REF_G_TWICEREAL 2 Re{G_{0}} @ f_{pcal} $(IFO):CAL-CS_TDEP_PCALY_LINE1_REF_ONEPLUSG_MODSQ |1+G_{0}|^2 @ f_{pcal} $(IFO):CAL-CS_TDEP_PCALY_LINE1_REF_CLG0_$(REAL,IMAG) C_{0} / (1 + G_{0}) @ f_{pcal} for Eq. 9 and Eq. 11 for f_{ctrl} @ ~ 35 [Hz] User entered EPICs records: $(IFO):CAL-CS_TDEP_DARM_LINE1_REF_OLG_$(REAL,IMAG) G_{0} @ f_{ctrl} $(IFO):CAL-CS_TDEP_DARM_LINE1_REF_D_$(REAL,IMAG) D @ f_{ctrl} $(IFO):CAL-CS_TDEP_DARM_LINE1_REF_C_$(REAL,IMAG) C_{0} @ f_{ctrl} $(IFO):CAL-CS_TDEP_DARM_LINE1_REF_C_NOCAVPOLE_$(REAL,IMAG) C_{res} @ f_{ctrl} $(IFO):CAL-CS_TDEP_DARM_LINE1_REF_A_TST_$(REAL,IMAG) A_{0}^{tst} @ f_{ctrl} $(IFO):CAL-CS_TDEP_DARM_LINE1_REF_A_PUM_$(REAL,IMAG) A_{0}^{pum} @ f_{ctrl} $(IFO):CAL-CS_TDEP_DARM_LINE1_REF_A_UIM_$(REAL,IMAG) A_{0}^{uim} @ f_{ctrl} $(IFO):CAL-CS_TDEP_DARM_LINE1_REF_A_TOP_$(REAL,IMAG) A_{0}^{top} @ f_{ctrl} Calcalulated records: $(IFO):CAL-CS_TDEP_DARM_LINE1_REF_A_TST_MODSQ |A_{0}^{tst}|^2 @ f_{ctrl} $(IFO):CAL-CS_TDEP_DARM_LINE1_REF_A_TST_NEGINV_$(REAL,IMAG) - (1 / A_{0}) @ f_{ctrl} $(IFO):CAL-CS_TDEP_DARM_LINE1_REF_A_USUM_$(REAL,IMAG,MODSQ) A_{0}^{pu} @ f_{ctrl} = A_{0}^{top} + A_{0}^{uim} + A_{0}^{pum} $(IFO):CAL-CS_TDEP_DARM_LINE1_REF_A_USUM_INV_$(REAL,IMAG) 1 / A_{0} @ f_{ctrl} This is the constant part of Eq. 12 $(IFO):CAL-CS_TDEP_DARM_LINE1_REF_C_MODSQ |C_{0}|^{2} @ f_{ctrl} $(IFO):CAL-CS_TDEP_DARM_LINE1_REF_INVCLG_$(REAL,IMAG) (1 + G_{0}) / C_{0} @ f_{ctrl} For Eq 9, Calculated Record: $(IFO):CAL-CS_TDEP_REF_INVA_CLGRATIO_TST_$(REAL,IMAG) - (1 / A_{0}^{tst})|f_tst * C_{0} / (1 + G_{0})|f_pcal * (1 + G_{0}) / C_{0}|f_tst This is the constant part of Eq. 9 For Eq 11, Calculated Record: $(IFO):CAL-CS_TDEP_REF_CLGRATIO_CTRL_$(REAL,IMAG) C_{0} / (1 + G_{0})|f_pcal * (1 + G_{0}) / C_{0} |f_ctrl This is the constant part of Eq. 11 for f_{pcal2} @ ~330 [Hz] (which are currently running on PCALY) User entered EPICs records: $(IFO):CAL-CS_TDEP_PCALY_LINE2_REF_OLG_$(REAL,IMAG) G_{0} @ f_{pcal} $(IFO):CAL-CS_TDEP_PCALY_LINE2_REF_D_$(REAL,IMAG) D @ f_{pcal} For Eq. 15 $(IFO):CAL-CS_TDEP_PCALY_LINE2_REF_C_$(REAL,IMAG) C_{0} @ f_{pcal} $(IFO):CAL-CS_TDEP_PCALY_LINE2_REF_C_NOCAVPOLE_$(REAL,IMAG) C_{res} @ f_{pcal} For Eq. 15 $(IFO):CAL-CS_TDEP_PCALY_LINE2_REF_A_TST_$(REAL,IMAG) A_{0}^{tst} @ f_{pcal} For Eq. 15 $(IFO):CAL-CS_TDEP_PCALY_LINE2_REF_A_PUM_$(REAL,IMAG) A_{0}^{pum} @ f_{pcal} $(IFO):CAL-CS_TDEP_PCALY_LINE2_REF_A_UIM_$(REAL,IMAG) A_{0}^{uim} @ f_{pcal} $(IFO):CAL-CS_TDEP_PCALY_LINE2_REF_A_TOP_$(REAL,IMAG) A_{0}^{top} @ f_{pcal} Calcalulated records: $(IFO):CAL-CS_TDEP_PCALY_LINE2_REF_A_USUM_$(REAL,IMAG) A_{0}^{pu} @ f_{pcal} = A_{0}^{top} + A_{0}^{uim} + A_{0}^{pum}, For Eq. 15 $(IFO):CAL-CS_TDEP_PCALY_LINE2_REF_G_MODSQ |G_{0}|^{2} @ f_{pcal} $(IFO):CAL-CS_TDEP_PCALY_LINE2_REF_G_TWICEREAL 2 Re{G_{0}} @ f_{pcal} $(IFO):CAL-CS_TDEP_PCALY_LINE2_REF_ONEPLUSG_MODSQ |1+G_{0}|^2 @ f_{pcal} $(IFO):CAL-CS_TDEP_PCALY_LINE2_REF_CLG0_$(REAL,IMAG) C_{0} / (1 + G_{0}) @ f_{pcal} for Eq. 9 and Eq. 11 I've not yet installed all of the above user defined EPICs records, bare with us while we get the MEDM infrastructure updated, and then we'll fill in the values needed. Note, the only difference from what I installed on Friday (see LHO aLOG 20291), is what was quoted for the "ESD_LINE" there is now called "DARM_LINE," and I need to install new values for the *actual* "ESD_LINE". Also, I've committed all model changes to the userapps svn repo, and I have captured all of the new new settings (thus far) in the SDF system, initializing new channels, and removing old channels.
JeffK, Darkhan
MEDM screens are undergoing changes to repliate recent front-end model changes (see attached screenshots). The screens are currently in the state at which we can input parameters needed for calibration: C_0, D_0, A_0^{tst}, A_0^{pum}, A_0^{uim}, A_0^{top}, line frequency and amplitude.
Current status of the screens have been committed to SVN (@ r11235). The modification process will continue tomorrow.
We also found a bug in front-end model that need to be fixed (using Im instead of Re part of a quantity, see attachment).
Not just lost communications as from 20333. This is an SEI system. If this system has only lost communications then sure do nothing. But that state should be confirmed. If it isn't actually running then more action is call for.
At the end station, the controller computers front panel heartbeat light is not flashing. The VFD Frequency output to the motor is not changing. These two indicators convince me that the control is in fact not running. The analog pressure gauge at the pump station is reading about 78 PSI. This indicates that we got lucky and the fixed output to the VFD is pretty close to what we need to maintain the ideal pressure. About 80psi out of the pump station is nominal. Looking at the trends of the pressures and motor drive, attached, it appears they latched below the average of the normal variation. So while the pressures are trending down (only based on where the trends froze and the analog gauge,) they are doing so at a slow rate and we could be lucky and make it to Tuesday.
The IFO just dropped and Evan gave me the okay to restart. The SSH failed though so a power cycle of the computer is called for. This will certainly take down the Pump Station and therefore the SEI platforms. So we elect to wait.
If the trend had been steeper, the next indicator would have been a tripping of the HEPI as the pressures dropped too low to drive the requested offset. Meanwhile the performance of the platform would likely have become weird as the actuators were attemping to work at a pressure outside their design. Had the trend been up rather than down, if the actuators were still working at 125psi, next the in-line pressure relief valve would have opened and closed repeatedly as the pressure exceeded and dropped below the threshold. Who knows if the HEPI platform would survive this but eventually enough fluid would have been lost to the pressure relief path that the reservoir fluid trip would abruptly trip the pump station and of course the platfrom, ISI, IFO and recovery would be more painful.
[Jenne, Gabriele]
It's been quite a struggle to get the IFO relocked today. Last night, Evan et al. ran a TCS tuning test, which left the TCSX CO2 power at 0.43 W. Yesterday, it had been at 0.23 W.
Just now (03:12:00-ish UTC) we returned the TCSX power to 0.23 W, and we're leaving it to re-equilibrate.
Separately, although we don't think this affected our ability to lock, the SYS_DIAG guardian is dead. It went into error at some point, and reloading it didn't clear the error. Stefan tried to restart it, but it never came back up. We await expert advice on fixing this.
It looks like (see log messages below) there is a syntax error somewhere, although no one (in the control room at least....) was editing it. Perhaps we entered some "if" loop that doesn't usually get run, so that's why this hasn't been caught before?
2015-08-09T00:33:40.06923 File "/opt/rtcds/userapps/release/sys/h1/guardian/SYS_DIAG_tests.py", line 403
2015-08-09T00:33:40.06929 yield 'ISI {} WD saturation count is greater than 75% of max'.format(chamber)
2015-08-09T00:33:40.06932 ^
2015-08-09T00:33:40.06939 SyntaxError: invalid syntax
2015-08-09T00:33:40.09600 guardian process stopped: 255 0
There were some typos in the SEI_STATE test that were preventing the node from restarting.
I destroyed and created the node (probably unnecessary), and then started it.
Also, SYS_DIAG_tests.py had not been checked in to the SVN for quite some time.
Thanks, Evan. It was definitely unecessary to destroy/recreate the node, but it doesn't hurt anything. Probably all you needed to do was just reload.
I looked into the glitches created by Robert's dust injections. In brief: some of the glitches, but not all of them, look very similar to the loud glitches we are hunting down
Here is the procedure:
In this way I could detect a total of 42 glitches. The last plot shows the time of each glitch (circles) compared with the time of Robert's taps (horizontal lines). They match quite well, so we can confidently conclude that all the 41 glitches are due to dust. The times of my detected glitches are reported in the attached text file, together with a rough classification (see below)
I then looked at all the glitches, one by one, to classify them based on the shape. My goal was to see if they are similar to the glitches we've been hunting.
A few of them (4) are not clear (class 0), some others (14) are somehow slower than what we are looking for (class 3). Seven of them have a shape very close to the loud glitches we are looking for (class 1), and 16 more are less obvious but they could still be of the same kind, just larger (class 2).
See the attached plots for some examples of classes.
It seems the text file of glitch times didn't make it into the attachments, would you mind trying to attach it again?
Ops! Here's the file with the glitch times.
Gabriele, Did you check which of Robert's glitches caused any ADC/DAC adjurations? The glitch shape will start changing significantly once the amplitude is big enough to start saturations. PS: The ODC master channel has a bit that will toggle to red (0) is any of the subsystems reports a saturation (with 16k resolution) - it might be exactly what you need in this case.
Stefan, I checked for some ADC and DAC overflows during this data segment. The OMC DCPDs ADCs overflowed during several of these. There were still some with SNRs of 10,000 that didn't overflow like this. The segments are pasted below. They're a bit conservative because I'm using a 16 Hz channel without great timing. There were no ADC overflows in POP_A, and no DAC overflows in the L2 or L3 of the ETMs. I didn't check anything else. This is not quite the same as what the ODC does, which is a little more stringent. I'm just looking for changes in the FEC OVERFLOW_ACC channels.
1123084682.4375 1123084682.5625
1123084957.3750 1123084957.5000
1123086187.3750 1123086187.5000
1123086446.8125 1123086447.3750
1123088113.0625 1123088113.1875
1123088757.4375 1123088757.6250
1123088787.1875 1123088787.3125
1123088832.3125 1123088832.4375
1123089252.6250 1123089252.7500
1123089497.2500 1123089497.3750
I injected particulate by tapping on the beam tube at various locations with a scissors, imitating the taps that the cleaners make by accident. The acceleration measured on the beam tube for similar taps was around 0.1 g with a frequency peak at about 1000 Hz (Link). At each location I made the first tap at the top of the minute and made a tap at every multiple of 5 seconds for 1 or 2 minutes. My tapping uncertainty was about 1 second.
To observe the glitches in DARM I filtered the time series of H1:CAL-DELTA_RESIDUAL_DQ to be dominated by the 120-1000 Hz band, with violin modes notched. The table shows the time and the fraction of taps that made glitches. The distribution of glitch sizes is shown in the Figure. There were roughly the same number of glitches in each size decade.
In summary:
1) Glitches were produced from most regions of the beam tube.
2) Only about 20% of taps produced glitches
3) 9/252 taps produced multiple glitches
4) Only 1 glitch of size comparable to the particulate glitches was observed more than 1s from a tap time in the entire 2 hour lock (DetChar may want to double check this). Thus the data suggest that there is not a reservoir of particles that are freed by the taps but fall later at an exponentially decreasing rate (so it is very unlikely that midnight glitches are particles freed by cleaning and cleaning is unlikely to have increased the background rate).
5) The figure shows that the number of glitches in each size decade was about the same, not increasing with smaller size.
|
Location on beamtube |
Time of first tap. Aug. 8 UTC |
Tap spacing (seconds) |
Duration (minutes) |
Number of taps |
Large glitches in DARM within 1s |
Large glitches not within 1s |
|
Y2-8 double |
15:52:00 |
5 |
1 |
12 |
4 |
0 |
|
Y2-2 double |
15:57:00 |
5 |
1 |
12 |
2 |
1, 2s |
|
Y1-2 double |
16:02:00 |
5 |
1 |
12 |
5 |
0 |
|
X2-8 double |
16:22:00 |
5 |
2 |
24 |
7 |
0 |
|
X2-2 double |
16:27:00 |
5 |
2 |
24 |
1 |
0 |
|
X1-2 double |
16:33:00 |
5 |
2 |
24 |
0 |
0 |
|
Y2-8 +1Y single |
16:44:00 |
5 |
2 |
24 |
0 |
0 |
|
Y2-4 +1Y single |
16:48:00 |
5 |
2 |
24 |
1 |
0 |
|
Y1-4 +1Y single |
16:53:00 |
5 |
2 |
24 |
13 |
0 |
|
X2-8 +1X single |
17:05:00 |
5 |
2 |
24 |
13 |
0 |
|
X2-4 +1X single |
17:12:00 |
5 |
2 |
24 |
5 |
0 |
|
X1-4 +1X single |
17:18:00 |
5 |
2 |
24 |
5 |
0 |
|
Number of taps that produced glitches (percentage) |
|
|
|
|
42 (17%) |
|
|
Total number of glitches |
|
|
|
|
56 |
|
This is very interesting. Can you sort and plot the raw time series of the larger glitches according to X vs. Y arm? (The attack should be unipolar and of opposite sign for the two arms.) It will also be interesting to note if there is a FWHM dependence on axial position.
I've attached a list of all Omicron triggers in the lock with SNRs above 100. The columns are the GPS time, peak frequency, and SNR. Rows marked with an X have an ADC overflow in the OMC DCPDs, so they may not be as useful for determining the shape of the glitches. Several of the glitches have very messy shapes in OMC DCPD. There's a few in the Y arm that have a fairly unambiguous single upward spike, for instance 1123084687.570. The whitened timeseries is attached. The shape is a triangle with a base of roughly 1 or 2 milliseconds. I haven't found anything in the X arm yet with a simple unambiguous shape, but I haven't checked everything.
Prior to this change, photodiodes (TX and Rx PD) calibartion coeffcient were reported in metres/Volt *(1/f^2). Now with the suspension model in place, we have calibrated the photodiodes in terms of Force Coefficient (N/V). The filter banks, as shown in the attachement above, now reflect these new N/V calibration factors. Appropriate changes have been made to the DCC document (T1500252) as well.
This is an additional note about the quack function -- when I was using quack, I had a difficulty in quacking an state-space suspension model into a foton filter. See the detail below.
In matlab, I had been using something like:
quad.d = minreal( zpk( c2d(quad.ss, smaplingTime, 'tustin' ), tolerance )quad.ss is a state-space representation of the quad suspension response, and samplingTime in this case is 1/16384 sec. The reason why I used the minreal function is that otherwise it would come with too many number of poles and zeros which exceeded the number of poles and zeros that foton can handle. I tried adjusting the tolerance of minreal in order to reduce the number of poles and zeros, but it was extremely difficult because it ended up with either too many poles/zeros or too few poles and zeros.quad.d = c2d( minreal( zpk(plant.ss), tolerance ), samplingTime, 'tustin');minreal before c2d. This allowed me for having a reasonable number of poles and zeros.I can't find the posts now, but several months ago, an intermittent issue with ETMX was spotted that was narrowed down to the CPS's, possibly specfically the corner 2 cpses (?). This problem then somehow "fixed" itself and was quiet for months. As of the night of the 4th, it seems to be back, intermittently (first attached image, spectr should be pretty smooth around 1 hz, it's decidedly toothy). Looking at the Detchar pages, it shows up about 8 UTC and disappears sometime later. I took spectra from last night (second image) and everything was normal again.
Still don't know what this is. Anybody turn anything on Monday afternoon at EX that shouldn't be?
I had turned on the NEG Bayard Alpert gauge at end X yesterday, but I have verified at least through Beckhoff that I turned it back off.