Following up on Kiwamu's report, here is the summary page for the coherences during the low noise lock:
https://ldas-jobs.ligo.caltech.edu/~gabriele.vajente/bruco_1110961816/
As usual, here is my digest.
In brief: we need MICH/SRCL subtraction, check SRC angular controls, they might be injecting noise, better tune the IMC longitudinal offset if possible
More details:
P.S. Why are you sending DARM signal into the CARM filter bank?
Ahh you caught on to our hacky audio setup, Gabriele. The OMC-DCPD --> CARM matrix element is enabled so that we can filter OMC-DCPD in the CARM bank before sending it to speakers in the control room. (Actually I'm not sure anyone is using this anymore, we can probably discontinue it.)
Also I have modified the ITMX violin mode damping so that there is 50x less gain at 450Hz. (There is an 8th order butterworth bandpass now, instead of a 4th-order.) This might make the noise a little worse between 480-510Hz, but I think that band is a lost cause anyway.
Commissioners were complaining of excess arm motion, so I've changed the blends on the beamline directions on St1, from the nominal 45mhz blends to 90mhz. To review for those not in the seismic know, this is done by opening the St1 Blend screen from the BSC ISI overview and clicking on the 90mhz blend in the appropriate direction (i.e. X for ETMX, Y for ETMY). Attached screens show overview with circled blend screen, St1 Blend overview, close up the blend X bank. It's not perfectly clear that we need to be using the 45mhz blend or if the 90mhz blend is good enough, but I've been talking to some of the commissioner about maybe doing an assessment when it's convenient.
They are switched back since the wind died down
Installation of the 5 channel ESD filter (D1500113) was completed at EY. We removed the existing Current limiting box and Bias filter box and replaced it with this new box making sure to reconnect the cables the same as before. We then restored the system and verified voltage could be applied to the ESD driver.
Some notes related to the low noise actions that will happen today. COIL DRIVER SWITCHING: - Coil driver state description - LLO log entry with description of switching of actuator states , Evan is looking at Den's script to do the BS actuator switching one coil at the time MICH/SRCL FEEDFORWARD: - Den's talk G1500281, page 11 (also attached): by implementing the feed-forward subtraction actuating on ITM L2, the feedforward paths for both MICH and SRCL are just a constant gain (MICH control is done through BS M2; SRCL is controlled by actuating on SRM M3, so you would need to filter the SRCL correction signal by ~f^2 to actuate on ITM L2, but the optical response goes as 1/f^2)
Following Keita's entry from last week, we are now running with the OMC in a lower-gain state to reduce the length actuation. First plot attached is a OLTF measurement of the OMC LSC loop while in low noise; the current Guardian settings bring us to the blue curve, with a UGF of about 15Hz. For the blue and green curves the OMC-LSC_SERVO_GAIN = 10, the red curve had a gain of 60 and a larger boost. We thought about turning off the boost entirely, which would give us the green curve -- for now we've left it on.
This gave us a good improvement in the upconverted noise around the OMC length dither line, see second figure which is the noise from late Friday night. (The areas of improvement compared to the blue reference, from Thursday morning, are annotated.) There is still some upconverted noise, see third figure, and we could reduce the OMC length gain even further.
To do: estimate the OMC length noise --> DARM coupling using Koji's noise measurements of the intrinsic length noise in the OMC to see how much length noise suppression we need.
PSL Status: SysStat: All Green, except VB program offline Output power: 31.9w Frontend Watch: Green HPO Watch: Red PMC: Locked: 5 days, 23 hours, 1 minutes Reflected power: 2.2w Transmitted power: 22.7w Total Power: 24.9w ISS: Diffracted power: 7% Last saturation event: 0 days, 8hours, 6minutes FSS: Locked: 0 days, 0 hours, 7 minutes Trans PD: 1.6v
SEI - hoping to get a new isolation filter installed on BS stage2.
SUS- nothing to report
CDS - ESD filters will be installed
3IFO - Tuesdat activities will include tagging and bar-coding of TCS and ISC items. Calum will be on site next week. Bubba is continuing to connect piping o containers for N2 purging.
FACILITIES - BTE cleaning is ongoing. Semi-annual lubrication of axial fans for the ventilation sytems will start today with the mid stations and may move to the corner statinos by today. Filters will NOT be removed.
TJ asked me about the BRS this morning, as it was tripping an alarm he had set. When I opened the overview, the numbers were rather large and swinging rapidly (over a minute it was going from -40000 cts to +40000 cts, the transition was a couple seconds. The attached dataviewer trend shows it has bee doing this since the 21st, starting about 15:00 UTC. No clue what happened.
A little bit more supporting data: I attach a time series of the drift monitor (the DC comparison between the autocollimator's fringe patterns for the balance mirror and reference mirror; in military green; H1:ISI-GND_BRS_ETMX_DRIFTMON) and control signal for the gravitational damping motor (in dark red; H1:ISI-GND_BRS_ETMX_DAMPCTRLMON). I suspect the suspension has drifted enough that fringe patterns have crossed causing the output of the sensor to go non-linear. This then rung up the suspension after the damper turned on thinking that the SUS had rung up. Then we get into a nasty feedback loop. Looks like the temperature in the XVEA has remained stable over the weekend (and for several days prior), so I doubt we can blame the temperature swings for the cause of the drift. For now, Jim and TJ have turned off the control software to let the suspension cool off. We'll do a more invasive investigation tomorrow morning.
I've attached a couple of plots showing the BRS_DRIFTMON channels. First one shows the 1-hr period when the BRS data started to grow large. It looks like some 'disturbance' set it off and then the damper took it away with a positive feedback loop because it was in the incorrect position.
The next plot shows the same channel over 3 days, starting from 2 days before the disturbance. It looks like the mean position was quite stable ( except for the odd missing dataset). Once the positive feedback loop got started, it drove the turn table harder and harder, generating heat and causing the mean position to shift slowly. Once things settle down, the mean position should return to ~ -2.9k counts. This is well within the linear range of BRS.
Due to the large amplitudes BRS reached before the software could be turned off, I'm not sure it will be sufficiently damped so it could be restarted tomorrow.
at 8:25 in an attempt to continue the locking sequence that had gotten "stuck" at engatge "ENGAGE_ASC". I selected "DC_READOUT_TRANSITION" to try and get it moving again. THis tripped the SUS RMS watchdogs on the ETMs. I was able to reset ETMY remotely but ETMX seems to need a PUM coil driver power cycle.
Filiberto power cycled ETMX PUM (and UIM accidentaly) and this cleared the UL RMS trip.
We should really just rip out these PUM driver RMS watchdogs, they're serving no protective purpose anymore -- unless there's something I don't know. It would be great if DetChar took a look at the many PUM driver watchdog trips that have happened over the past few weeks (using ${IFO}:SUS-${OPTIC}_BIO_L2_MON as the readback, an EPICs channel, so it should exist in the frames), and find out when the current goes over using the RMS current monitor, ${IFO}:SUS-${OPTIC}_L2_RMSIMON_UL_MON (and EPICs channel, so it should exist in the frames), and see if that corresponds to some actually dangerous level of current to the internal circuitry. If not, we rip this pesky thing out. Recall you can find all the information you could need in page 3 of the QUAD controls design description: T1100378. The only trickery that is poorly documented is that the binary input readback, ${IFO}:SUS-${OPTIC}_BIO_L2_MON, actually contains many status bits. Whether the UL, LL, UR, LR RMS current watchdogs are tripped are readback bits 2.^[1 5 9 13], which means you need to mask the bit word with the numbers 2, 32, 512, and 8192 to obtain the right information. The best "source" for this information is the QUAD's MEDM screen overview which has these masks, in concert with the first page of PUM driver's schematic, D070483.
model restarts logged for Sat 21/Mar/2015
2015_03_21 14:42 h1fw0
model restarts logged for Sun 22/Mar/2015
2015_03_22 11:28 h1fw1
both unexpected restarts.
The CDS web server cdswiki was dead this morning, and needed a reboot. It started normally, no explanation for the failure.
Evan, Lisa Evan started locking early in the afternoon. He collected 10+ successful locking sequences in a row without manual adjustment (WOW!), see first plot. However, in order to keep the interferometer locked on the operating point, he had to open most of the alignment loops, keeping closed only DHARD and BS. We realized later that it was due to some low frequency oscillation in the CHARD loop, which was triggered by closing the SRM/PRM loops. We decided to keep those loops opened for now. We then realized that the power build up has been consistently low for a while (second plot - the beginning of the plot corresponds toMarch 9 , when a manual adjustment of the ITMs brought the arm power to its maximum ~ 1100 counts), so we decided to go through the initial alignment and re-look at the ITMs position..and an earthquake happened.. P.S.: All of the above happened with DARM locked on RF.
This is a better plot with the trend of the arm power over the last 45 days. The build up started to be consistently low after the first week of Mach, with some attempts of bringing it back (but resulting in a not stable IFO, I am told).
Dan, Kiwamu,
Some reports from last night and a little bit from today.
(Non stationary behavior of noise floor in 20-100 Hz)
Last night we worked more on the L2 actuation in order to see what noise are below the ESD DAC noise. The recycling gain was as high as 27 and we could smoothly fully lock the interferometer multiple times. After we worked on the OMC trans fluctuation issue (see Dan's report), we transitioned to ETMY from ETMX. The L1 and L2 stages of ETMY were used. As we saw last time (alog 17367), the DARM spectrum showed a non stationary behavior in 20-100 Hz frequency band when small bias and quadrant voltages (< 100 V) were requested. The noise floor was breathing up and down on a time scale of ~10 seconds. We varied the ESD bias voltage to see if there is a "sweet spot", but because of the non stationary noise floor, we were unable to find it. On the other hand, having +380 V on ETMX ESD consistently brought us back to the previous noise floor. So this agrees with the hypothesis that we were limited by the ESD DAC noise. A funny thing is that changing the ETMY ESD bias did not seem to make the noise floor better or worse -- even with +380 or -380 V on the ETMY bias did not bring the noise floor visibly high.
At one point we noticed that the noise floor seemed to be correlated with DHARD YAW -- when the DHARD YAW deviates from the operating point, the noise floor in DARM seemed to go up. So we newly engaged a boost (to suppress 100 mHz variation) in DHARD YAW which seemed to somewhat reduce the non stationary behavior. We need to investigate this issue a bit more.
Here are some time to look at:
I attach a DARM spectrum starting at Mar-21-2015 8:30:00 UTC.
(Unable to damp ITMX roll mode)
Today I happened to be in the control room and so started locking. The recycling gain was as low as 24 today even though I went through the initial alignment. After reaching full resonance with RF darm, the roll mode of ITMX at 13.81 Hz was found to be as high as 10-12 m/sqrtHz in the DARM spectrum again (alog 17293). Initially I tried the same damping setting on ITMX, but I was unable to damp. Then I even tried the minus sign and +-60 deg phase rotators, but did not have a good luck. I wonder if this is related to the fact that the initial alignment today was far from optimum (i.e. low recycling gain).
Dan, Sheila, ITMX
While testing the vioin mode damping tonight, we had an abrupt lockloss, and for a short period my bandpass filter at 501.094Hz was sending noise x120dB into the ITMX L2 coils. This tripped the coil current watchdog, just like a previous event on ETMY a few weeks ago. (There are now limiters on all the violin mode damping filters...)
Unlike ETMY we can't reset this watchdog by toggling the RMS Reset epics channel. Anyone know how to clear this error?
We suspect something in hardware is preventing a software reset. In the CDS highbay, the SUS Independent Watchdog electronics box has red lights that won't reset when we use the 'Fault Reset' button.
I asked the question about how to clear it when I was designing an alert on the Ops Overview screen for it. As I recall, all you have to do is change the value in the field from a 1 to a 0 and then back again? Also, I think Stuart Aston told me that watchdog would eventually go away.
So, normally changing that value back and forth WOULD be the way to clear it according to Stuart. This did not work. He then suggested I simply power cycle the PUM so with Sigg's approval I did and that seems to have worked. The L2 RMS watchdog trip on ITMX has been cleared. On a side note, this 'SUS Ind WD' chassis is ONLY cabled to the BIO I/O chassis for ITMY. It IS NOT documented on any of the current drawings and it isn't commissioned. It's current state can't be reset with the button on the front so I assume that it would have to be power cycled to be cleared. That being said, it isn't really connected to anything.
Currently experiencing the same behavior with ETMX L2 UL. Toggling the RMS WD reset does nothing.
Also, the ETMX indicator on the ops screen does not indicate this fault (the border is still green).
[Stuart A, Jeff K] As was recently carried-out at LLO (see LLO aLOG entry 16663), during this mornings maintenance period I was able to roll-out independent switching of BSFM M2 stage coil driver BIO filter states, as outlined in ECR E1500045. Hopefully, this should provide commissioners with the capability to smoothly (without breaking lock) transition from acquisition to the low-noise state coil driver, just for the Beamsplitter M2 stage, which was problematic at LLO. This was implemented as follows: (1) Svn'd up the common library parts: /opt/rtcds/userapps/trunk/sus/common/models/ U STATE_BIO_MASTER.mdl U BSFM_MASTER.mdl (2) Svn'd up the common MEDM screens: /opt/rtcds/userapps/trunk/sus/common/medm/bsfm/ U SUS_CUST_BSFM_OVERVIEW.adl U SUS_CUST_BSFM_BIO.adl U SUS_CUST_BSFM_M2_EUL2OSEM.adl (3) Rebuilt, installed and restarted the h1susbs model without incident. (4) Noticed some white boxes on EUL2OSEM Ramping Matrix MEDM screen, which were site specific to L1, so I fixed these with a $(IFO) substitution and re-committed SUS_CUST_BSFM_M2_EUL2OSEM.adl to the svn. (5) 4 new EPICS channels are present for the M2 BIO states, as well as an additional 4 for the test coil enable, all which need to be configured: - All coil enables were set to 1, and the BSFM M2 BIO state was set to the 'default' of 1, as I was informed by Jeff K was being used by commissioners n.b. LLO 'default' acquisition is BIO state 2, and low-noise is BIO state 3 (see LLO aLOG entry 16565). Screen-shots below show an example use case for the EUL2OSEM Ramping Matrix. with green indicating a nominal state i.e. set values and current value are equal, red when a new value has been set, and yellow after activating the LOAD MATRIX ramping in over the required duration. Finally, I took a new safe SDF snapshot using the same front-end technique I use at LLO: - Transition the Suspension to a SAFE state via Guardian - On the SDF_RESTORE MEDM screen available via the Suspension GDS_TP screen select FILE TYPE as "EPICS DB AS SDF" & FILE OPTIONS as "OVERWRITE" then click "SDF SAVE FILE" button to push a SDF snap shot to the target area (which is soft-linked to userapps). - This safe SDF snapshot was then checked into the userapps svn: /opt/rtcds/userapps/release/sus/h1/burtfiles/ M h1susbs_safe.snap This should close ECR E1500045 and Integration Issue #1003 for LHO.
LLO scripts for transitioning the BSFM M2 coil driver from acquisition (BIO state 2) to low-noise (BIO state 3) once IFO RF LOCK is archived have been checked into the repo and svn'd up here at LHO: /opt/rtcds/userapps/release/lsc/l1/scripts/transition/als/ A bs_m2_switch.sh A bs_m2_out_normal.snap A bs_m2_out_ll.snap A bs_m2_out_lr.snap A bs_m2_out_ul.snap A bs_m2_out_ur.snap The above script applies a BURT snapshot to the BSFM M2 EUL2OSEM Ramping Matrix zeroing a quadrant/channel at a time using the other 3 to actuate while it's state is transitioned. Once transitioned the script moves on to the next quadrant, until all 4 are complete. This functionality should be incorporated into a Guardian transition state.
I copied this scripts and the burt files into userapps/release/isc/h1/scripts/sus
, replaced L1
with H1
throughout, and committed them to the SVN.
The script seems to work fine in terms of writing the right values to the right channels. We haven’t tried it yet with the interferometer locked.