BRS software crashed on the 25th, so today I went to EX to restrart the code. I kept the damper commented out, so the damper is currently OFF. I will turn it back on when it seems like it will be calm down at EX for a bit (vac team is down there right now).
Within a min afer a lockloss, the ISI ETMY ST1/2 WDs tripped. The medm said that it was ST1 T240 and the plots showed a slow drift to the WD trip level (plot attached). HEPI moved 170 um after lockloss before tidal began to bleed down. A further investigation is ongoing by the SEI team.
Looks like this is a trip we will suffer whenever the drive to the HEPI is large enough at the time of lockloss. That doesn't necessarily mean the length of lock but it is certainly related. If the tide turns around and the offset given to HEPI heads back to zero, we could be fine after a long lock stretch.
The attached 30 min second trend plots are around the three most recent locklosses from some low noise state. In each case you can see the T240 tilting towards trip level but only the most recent trip was the tidal offset large enough to 'tilt' the T240 long enough to hit the trigger. The HEPI output from ISC is in nm.
Is the problem the bleed off too fast? It seems this is 2um/sec. Is there horizontal to tilt coupling that needs to be addressed? Is it a problem anyway? The T240 will likely take a couple minutes at worse to settle and another minute for the ISI to reisolate.
CDS: EE shop work to prep for Ring Heater installation PSL: Wants to tweak PR3 OpLev at first available opportunity FAC: Continued beam tube cleaning Y-Arm VAC: Moving pump cart to End-X for weekend long RGA bake out
Evan, Matt, Lisa We took ~25 min of undisturbed data starting at Jul 31 2015 09:21:13 UTC , after the MICH FF improvement (and all the changes described earlier implemented), with the interferometer locked in low noise ~ 65 Mpc (according to SensMon). We saw a big glitch during this period (~9:28), a huge one right before (~9:18, right after the mich tuning was done), and another one at the end of the undisturbed period. We went back to commissioning mode as Evan is starting some noise injections for his noise budget.
I did injections into MICH (from 10:41:40 to 10:47:00) and SRCL (from 11:15:00 to 11:22:40) with all FF on.
About 30 s after turning off the SRCL injection, a number of photodiode signals associated with the AS port dropped suddenly by about 5% (in particular: AS DC, AS90, ASA sum, ASB sum, ASC sum, and AS45Q; notably, the OMC DCPDs, POP DC, MC2 sum, and IMC input power did not see this). After that, the interferometer glitched for about 30 minutes. This event happened at 11:23:31.05 and lasted for only a few milliseconds.
I did an injection into PRCL (from 12:21:50 to 12:29:00).
I did an injection into the ISS around 12:52:00. I think this measurement needs to be redone by injecting into the error point of the outer loop rather than the inner loop. With the outer loop closed it is now difficult to get good coherence between intensity and DARM below 300 Hz.
PSL Peri PZT servo ON/OFF test:
IFO was still locked when I came in in the morning, so I briefly switched the IMC WFSB YAW to PSL PZT feedback path at around 7:50 AM local time.
In the attached, references are with the servo on, current traces are without the servo.
Middle finger (300Hz bump) is greatly reduced in OMC DCPD, but the coherence for right and index finger slightly increased due to gain peaking (expected), and there is another gain peaking at 620Hz (again expected). The last one could be reduced further by adding a deeper notch but that might increase the 350Hz bump gain peaking a bit.
The servo was turned on again at about 8AM+-5min.
Commissioning Team For the meeting tomorrow, here is a list with the things that we changed this week, and they are now part of the baseline locking sequence: 1) more aggressive cut-off in MICH, to reduce coherence with DARM above 50 Hz (log 20020 ); 2) more aggressive cut-offs in the ASC_AS_A / AS_AS_B --> OM1 / OM2 centering loops (from 100 Hz to 20 Hz) to remove some coherence with those signals reported by Bruco; 3) more aggressive cut-offs in the OMC SUS alignment loops ( log 20087 ) to remove unwanted OMC length motion which was observed to cause fringe wrapping by Josh et al (log 20079 ); 4) heroic modification of the DHARD loops (both PITCH and YAW) during the locking sequence, aimed to make the transition to REFL/TR and carm offset reduction more stable by increasing the phase margin of this loop, which was kind of poor and was seen to cause several lock losses in the past (log 20084 ); these loops have now high ugfs (~ 5 Hz) and >30 degrees phase margin. Note that the ASC work is not done yet: tomorrow we will add cut off filters, and attack CHARD. Also, we haven't yet succeeded in turning off the ITMs optical levers, which are still on; 5) modification of the ISS loop to improve phase/gain margin (log 20088 and log 20088 ); 6) IMC WFS --> PSL PZT loop to reduce ~300 Hz peaks in DARM ( log 20051 ); 7) found and fixed a sneaky ramp time problem which was causing the switch from ESD ETMX --> ETMY to fail repeatedly (log 20076 ), Jenne got a BIG PLUS for this one; 8) added an offset to PR3 during the CARM offset reduction to counteract the wire heating ( log 20055 ) 9) the Guardian usercode has been modified to speed up the locking sequence ( log 20055 ), and improve IMC_LOCK; The interferometer has been happily locked in low noise at ~ 65 Mpc for nearly 2 hours with all the above changes implemented, and the Guardian has been updated. 10) (Later edit) Evan just optimized the MICH feed forward, and he reduced the MICH coupling between 30 and 50 Hz; the sensitivity is now better in that region. Other things in our to-do-list for tomorrow: 1) More alignment work (cut-offs and CHARD) 2) Test the whole sequence again to check robustness 3) Track down the TMSX weirdness (log 20078) 4) Noise investigations and injections for noise budget
Sudarshan, Matt
The new ISS filter was seen to oscillate at ~1MHz with no input signal. This was fixed by adding 50pF cap into the feedback of the output opamp. This cap, in combination with the 10k resistor, puts a pole at 300kHz, which is fine for this loop. The second loop is currently closed (with the IFO at high power) and appears to be working properly.
Also, independently of the repair that Matt desribed above, we found that an op-amp in the very first stage kept railing at -10 V. It is AD620 stuffed in U10 of the 2nd loop servo box (D1300439-v2). We have no idea when this op-amp had died. Swapping the op-amp with a spare in the EE shop fixed the issue. We will update the e-traveler accordingly.
Evan, Matt, Josh S. (in spirit)
The OMC ASC loops now have 0.5Hz cut-off to remove unwanted OMC length motion (see plot). This change is intended to reduce the OMC SUS velocity so that we don't get fringes in DARM.
The filters are in OMC-ASC_{POS, ANG}_{X, Y} loops (see screenshot), and do not appear to cause stability problems (because the UGFs are below 100mHz) with the MASTER_GAIN up to 0.2, though some gain peaking is observed at this gain. The normal operating gain is 0.1.
Jenne, Matt, Sheila, Evan, Stefan, Lisa
Tonight we reworked DHARD YAW. We started with a new filter desgin, which avoids inverting any thing in the plant (our old design had just the 2-3Hz resonances inverted). The first screenshot shows the new control filter (which is in FM6). This filter gives us significantly better phase margin than we had before. We engage it in the state DARM WFS (right after transitioning to RF DARM) with a gain of 50, and leave the gain at 50 as the optical gain increases until the state CARM_5PM, where we reduce it by a factor of 2. This is because the sensor noise is too large at high CARM offsets to run with the final bandwidth. The attached screen shot shows the open loop gain at some different arm powers. We are not currently using a boost, but we think that we should be able to turn on the soft boost in FM3 (boostMS).
We also restored the DHARD PIT gain changes durring the CARM offset reduction that we found alst night, they seem to work fine.
After a half hour at 24 Watts with the new loop on and no ITM oplev damping, we saw a pitch instability ring up (about 6:55 UTC July 31st). This was fixed by turning back on the ITM PIT oplev damping. The oplev damping is back in the guardian for now.
This is to remind us that we haven't yet engaged a cut-off for PITCH. I made a new EPL33 filter to replace the current unused DHARD cut-off (ELP5,10) , such as this new cut-off can be engaged with the current higher DHARD ugf. I haven't loaded this filter yet.
Matt, Evan
Why do the TMSX RT and SD OSEMs have such huge spikes at 1821 Hz and harmonics? These spikes are about 4000 ct pp in the time series. In comparison, the other OSEMs on TMSX are 100 ct pp or less (F1 and LF shown for comparison).
Also attached are the spectra and time series of the corresponding IOP channels.
On a possibly related note: in full lock, the TMSX QPDs see more than 100 times more noise at 10 Hz than the TMSY QPDs do.
From Gabriele's bruco report, the X QPDs have some coherence with DARM around 78 Hz and 80 Hz. A coherence plot is attached.
It seems similar to the problem from log 12465. Recycling AA chassis power fixed the issue at the time.
Quenched the oscillation for now (Vern, Keita)
We were able to clearly hear some kHz-ish sound from the satellite amplifier of TMSX that is connected to SD and RT. Power cycling (i.e. removing the cable powering the BOSEM and connecting it again) didn't fix it despite many trials.
We moved to the driver, power cycled the driver chassis, and it didn't help either.
The tone of the audible oscillation changed when we wiggled the cable on the satellite amp, but that didn't fix it.
Vern gave the DB25 connector on the satellite amp a hard whack in a direction to push the connector further into the box, and that fixed the problem for now.
A little piece of script, deModMonitor.py, which helps you monitoring demodulated signals, is available at
/opt/rtcds/userapps/release/isc/h1/scripts/demodMonitor.
An example of how this code can be used is TCS_deModMonitor.py and run_TCS_deModMonitor.sh. The main purpose is to monitor how omc dcpd's response w.r.t. intensity noise (, or frequency noise or src length noise) changes when we heat up the compensation plates. A sinusoidal excitation is generated using Chris Wipf's awg module and injected to PSL-ISS_TRANSFER1_INJ. The code then reads out OMC DCPD's demodulated signal every certain amount of time and records the data to a txt file. In case that the code needs to be terminated during a measurement, it will catch ctrl+c and turn off the excitation.
The code is designed s.t. it should be able to used in a general demodulating-monitoring process.
Josh, Daniel Vander-Hyde, Kimberly Mercado
Kyle, Gerardo 0900 hrs. local Added 1.5" O-ring valve in series with existing 1.5" metal valve at Y-end RGA pump port -> Valved-in pump cart to RGA -> Valved-in Nitrogen calibration bottle into RGA -> Energized RGA filament -> Valved-out and removed pump cart from RGA -> Valved-in RGA to Y-end ???? hrs. local Began faraday analog continuous scan of Y-end 1140 hrs. local Isolated NEG pump from Y-end -> Began NEG pump regeneration (30 min. ramp up to 250C, 90 min. soak, stop heat and let cool to 150C) 1410 hrs. local Valved-in NEG pump to Y-end 1430 hrs. local Valved-out Nitrogen cal-gas from Y-end 1440 hrs. local Valved-in Nitrogen to Y-end -> Stop scanning
Plot of pressure at Y-End station before, during and afer NEG regeneration.
Response of PTs along the Y-arm to NEG pump regeneration.
RGA and pressure data files for NEG regenerations to be centralized in LIGO-T1500408
Interesting! As you predicted, the RGA is not super conclusive because of the background; but there seems a clear difference when you isolate the N2 calibration source. So your water and N2 may really be comparable to the hydrogen, say several e-9 torr each (comparing sum of peaks to the ion gage). The NEG will poop out after ingesting ~ 2 torr-liters of N2, so at 1000 l/s it will choke and need regen after a few weeks. Which is I guess what it did. It would be good to clean up the RGA so we can home in on the N2 and water pressure, and especially HC's (I expect the HC peaks in these plots are from the RGA itself). To get practical use out of the NEG we should pace how much of these non-hydrogen gases it sees. We can expect to only get about 50 regens after N2 saturation, and small amounts of HC may kill it outright. We should be able to estimate the net speed of the NEG before and after from the pressure rise and decay time (we can calculate the beam tube response if we presume it's all H2).
I have trouble seeing even the hydrogen pumping by the NEG by looking at the different scans. Suggest you set the RGA up to look at AMU vs time and do the leak and pump modulation again. Plot amu 2, amu12,amu14,amu28.
Rai,
That is on our list of things to do - make a table of the relevant amus' partial pressures.
Note that all the ascii data is at:
(see LIGO-T1500408-v1 for ascii data)
caution - 15 mbytes
Kyle can probably fish out the relevant data from the RGA computer so no need to plow through the whole file.
thanks for the comments, Mike and Rai.