Not much eye-catching here except ETMx. ETMx has taken a step over +/-10um in the last few hours for PITCH & YAW today (9/6).
Here is the timeline of what happened this morning at EX (see attached minute trend plot, showing time span 03:00 - 10:00 PDT).
At 03:28 h1iopsusex lost track of its DAC channels, and went into the safe state of not driving any DAC outputs. At this point TMSX and ETMX are not being damped, but the ISI is still operational (red line jumps to 132 in plot). Over the next hour, TMSX slowly rang up and eventually hit the software watchdog limit of 110mV RMS. Ten minutes later the SWWD tripped h1iopseiex DACs (puple line drop to zero) stopping the ISI isolation, which then quietened the suspensions down over the next three hours. When the control room untripped the SWWD at 08:28 the sequence repeated itself. The problem was resolved by restarting the h1iopsusex model at 09:43.
The initial problem of the IOP model losing track of its DAC channels has been seen before, and is more likely on the end station SUS machines due to their increased ADC glitch rate (faster computers).
Bottom line is the software watchdog acted correctly, and in this case resolved the ring-up of TMSX. For O3, the hardware watchdog will be active at EX as a fall-back watchdog in case the SWWD becomes non-functional.
Sheila restarted the models on h1iopsusex to fix the problem.
The clean room curtains were let to free hang so as to improve particle counts during the installation then removal of the Surface Discharge Ionizer assembly at the end stations as part of the TMDS exercises. As found, they had been clamped aside and prevented from touching the ISCTs. Is this an issue for commissioning this week?
End X HEPI, ISI & TMS were tripped upon arrival. SWWD was tripped (see attached screenshot). Reset TMSX DACKILLIOP, then SEIETMX DACKILLIOP, then HEPI WD, then ISI WD. Guardian brought everything back. CDS overview is reporting red blocks for DAC and DK on H1IOPSUSEX (see attached screenshot). Have not figured out how to clear these.
GDS screen for H1IOPSUSEX after pressing Diag Reset and OVERFLOWS.
SUS TMSX IOP DACKILL just tripped.
Daniel, Keita, Sheila, Thomas, Hang
1,
Because the 118.3MHz and 9.1MHz&45.5MHz SBs' phase are not sync'ed perfectly, we need to use a double demodulation scheme to sense the AS72 signal, see T1700324. Previously, we offset the first demodulation frequency by 205Hz (demod at 72.8M + 205 Hz for the first demod). However, with this config we could not engage the PLL loop to stabalize the secondary demod freq.
Daniel pointed out the if we domod at 72.8M + 205, we would down convert both <45.5M, 118.3M> and <2x45.5M, 2x9.1M> to 205Hz, and the two signals could beat against each other since 118.3M SB's phase drifted relative to 9.1M. Therefore the PLL loop could not be locked.
To solve it, we instead demod'ed at exactly 72.8M for the first demod loop, but offset the 118.3M by 205Hz. After this modification we could have a stable PLL loop to derive the 72.8M Hz signal.
2,
We preformed some preliminary AS72 vs AS36 comparison. Under DRMI config (10 W input),the SRM pitch respose in cnt/rtHz were (also the first plot)
| A_I | A_Q | B_I | B_Q | |
| AS36 | 253 | 207 | 944 | 360 |
| AS72 | 2.5 | 0.19 | 1.4 | 1.2 |
In order to see the signal, we had to have a very narrow bw of 0.01Hz. For reference, the noise level for AS72 is about ~0.1 cnt/rtHz.
Then in the NLN w/ full IFO, we did the measurement again (second plot)
| A_I | A_Q | B_I | B_Q | |
| AS36 | 318 | 77 | 786 | 223 |
| AS72 | 1.6 | 0.39 | 1.1 | 0.7 |
The noise level for AS72 is still about 0.1 cnt/rtHz.
In addition we report the sum signal under the full IFO
| A_I | A_Q | B_I | B_Q | |
| AS36 | 25 000 | -4 300 | 33 000 | -14 000 |
| AS72 | 169 | -3.3 | 211 | -1.8 |
We also tried to see the BS repose but could not see anything even if the line we drove dominated the BS rms motion...
Note that the 72 MHz signals is tiny because the modulation index for the 118 MHz is puny. See alog 37042. Since we are adding the new modulation to the EOM for the IMC, it is completely off-resonance. Obviously, we would need a dedicated EOM, if this installation should become permanent.
Patrick, Daniel, Hang, Sheila, TVo
We are back at NLN with a range of about 55 MPC.
Good news, it seems like the ETMY TMDS slightly lowered the noise floor in the 15-55 Hz band!
Attached is a spectra comparing a few different times, the first 3 are just to get a rough reference about what the noise floor was before the discharge measurement but after the Montana EQ.
It seems like ETMX discharge also helped a bit but it wasn't obvious unless you take on the order of hundreds of averages, this still doesn't reconcile all of the mystery noise but there is some progress.
We also looked at the spectra before/after discharging with jitter noise removed. Instead of running Jenne's code with time-domain causal filters, we just did a rough freq domain subtraction. The results are attached.
We'll try to check the calibration in the morning.
J. Kissel, S. Dwyer, T. Vo, Just in case any one is mistrusting that the changes in actuation strength of the ETMY from discharging were not covered by the time-dependent correction factors (i.e. kappa_TST), we show a zoom of the ~35 Hz PCAL calibration line. The largest discrepancy between line heights is 5%, which is within the stated limit of uncertainty and systematic error for 68% confidence interval. So, no need to mistrust the calibration here. We've decided against making a full sweep given that we're so crunched for time on this last night with the O1-O2-like H1 interferometer. May we come back more sensitive than ever!
Results attached. If you made one of these changes please commit it with a description of the reason the change was made.
Kyle, Gerardo Today we leak tested the new 2.75 CFF, valved-in the RGA and NEG pump, dumped GV18's unpumped gate annulus volume into the adjacent (pumped) annulus volume and opened GV18. This completes WP #7139 and #7139. Attached is the pumpdown. Note: the controller for the Y2-8 ion pump is showing 5600V, 1.0 x 10-11 Torr and 0.0 microamps pump current - hmmm. I STOPPED then STARTED the HV but get same reading. Neither the "Torr" nor the "current" are as expected. This might be related to the controller setup parameters. Otherwise, the pump isn't actually pumping! I'll consult Gerardo tomorrow.
Another ion pump cable issue?
What does X2-8 read?
Yes - the ion pump current could be at the limit of the controller's measurement circuit. Pressures on the two arms look to be the same now. I thought there was a factor of 10 difference yesterday but maybe I had the wrong glasses on. Kyle or Gerardo?
I checked the display of the controller for X2-8 this morning and found it displays the same as the Y2-8 controller. That's great and all and, obviously they are pumping, but I thought that we had observed leakage current in 300' HV cable that amounted to micro-amps. Also, as a general rule of thumb, you figure 1 micro-amp of pump current for a 100 l/s ion pump @ 1 x 10-9 torr. As such, we should be seeing 5 - 10 micro-amps of pump current in addition to any leakage current.
I suggest turning off one or both ion pumps and looking for a response on the vacuum gauge over a period of a day or more. Hopefully this would confirm that they are pumping. As to leakage current - maybe things have dried out significantly over the summer and therefore leakage has fallen? Need to get some of these signals into epics to enable trending.
TITLE: 09/06 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC STATE of H1: Commissioning INCOMING OPERATOR: None SHIFT SUMMARY: TMDS at end Y done. GV opened. Charge measurements run but not complete. Ran through initial alignment. Had to skip initial alignment of Y arm, something kept pushing the lock off. Sheila destroyed a guardian node that was interfering with locking ALS DIFF. Back to NLN at ~56 MPc. LOG: 14:40 UTC Peter working in back of optics lab 14:41 UTC Restarted video0 15:04 UTC Set observatory mode from maintenance to commisioning Jim running excitation on HAM4 15:49 UTC TJ to optics lab to retrieve item 16:53 UTC Kyle opening GV 17:03 UTC Peter done 17:22 UTC Richard to end Y to turn on high voltage for ESD and ring heater 17:44 UTC Richard done 19:45 UTC Starting locking 19:46 UTC Peter to optics lab 20:33 UTC Kyle to LVEA 20:51 UTC Kyle back 21:05 UTC Peter done 21:28 UTC Kyle to end Y to check on pump (adjacent room to VEA) 22:01 UTC Kyle back 23:44 UTC NLN ~56 MPc
Were not being used and will be replaced with hardware.
The ISI CPS noise spectra plots all look OK. The big rise/dip in the BRS-Y plot was Jim recentering the BRS. Close FAMIS task #6914.
Sheila plans to come in and lock the IFO this afternoon, so that I can try to finish these detchar safety injections. I've schedule 7 injections, the first one will begin at 16:06:22 PDT, and the last at 16:51:22 PDT. Here is the update to the schedule:
1187824000 H1 INJECT_DETCHAR_ACTIVE 0 1.0 detchar/detchar_27July2017_PCAL_{ifo}.txt
1187824450 H1 INJECT_DETCHAR_ACTIVE 0 1.0 detchar/detchar_27July2017_PCAL_{ifo}.txt
1187824900 H1 INJECT_DETCHAR_ACTIVE 0 1.0 detchar/detchar_27July2017_PCAL_{ifo}.txt
1187825350 H1 INJECT_DETCHAR_ACTIVE 0 1.0 detchar/detchar_27July2017_PCAL_{ifo}.txt
1187825800 H1 INJECT_DETCHAR_ACTIVE 0 1.0 detchar/detchar_27July2017_PCAL_{ifo}.txt
1187826250 H1 INJECT_DETCHAR_ACTIVE 0 1.0 detchar/detchar_27July2017_PCAL_{ifo}.txt
1187826700 H1 INJECT_DETCHAR_ACTIVE 0 1.0 detchar/detchar_27July2017_PCAL_{ifo}.txt
The IFO should be left out of observation mode. Also, I'm not sure if the PCALX 1500Hz line gets turned back on, but this also needs to be turned off. Simply typing the following on the command line will do the trick:
caput H1:CAL-PCALX_PCALOSC1_OSC_SINGAIN 0
caput H1:CAL-PCALX_PCALOSC1_OSC_COSGAIN 0
The interferometer is locked in Nominal low noise and the PCAL X 1500 kHz line is off as of 22:55 UTC. I hope that the BS ISI will not trip during the injections. (alog 38382)
The interferometer unlocked just before 1187825964, so I think that 4 or 5 of these injections would have completed before it unlocked. I don't know why it unlocked.
According to the guardian log (see attached), the first 5 of these injections made it in. The fifth one ended at 1187825913 (about 50secs before the lockloss). This should be enough injections for now. Thank you Sheila for coming in on a Saturday afternoon and locking the IFO for us!
Short version: I made omega scans for the 5 sets of injections that were done above andyou can find them here: https://ldas-jobs.ligo-wa.caltech.edu/~jrsmith/omegaScans/O2_detchar_injections/ (note that after the first set the rest of sub-directories of the start time for each set).
Long version: The Injection files are here: https://daqsvn.ligo-la.caltech.edu/svn/injection/hwinj/Details/detchar/. These are timeseries sampled at 16384 Hz and they start at the injection times listed in the original entry above. Times of the injections start 0.5s after times given in alog, and are separated by 3s each. Some simple Matlab to plot (attached) the injection file and print a list of times is:
% Plot hardware injections
fnm = 'detchar_27July2017_PCAL_H1.txt';
data = load(fnm);
t = 0:1/16384:(length(data)-1)*1/16384;
figure;
plot(t,data)
xlabel('time [s]')
ylabel('amplitude [h]')
title(strrep(fnm,'_','\_'))
orient landscape
saveas(gcf,'injections.pdf')
% Print times of injections
t_start = 1187824000+0.5;
times = t_start:3.0:t_start+106;
fprintf('%.2f
',times)
To submit the omega scans I used wdq-batch like so: (gwpysoft-2.7) [jrsmith@ldas-pcdev1 O2_detchar_injections]$ wdq-batch -i H1 times-1.txt, then submitted the resulting dag to Condor.
OmegaScan results are here: https://ldas-jobs.ligo-wa.caltech.edu/~jrsmith/omegaScans/O2_detchar_injections/
These will take some hours to run and we will then look at the results to see if there are any obvious unsafe channels. In addition we'll look statistically using hveto.
This is the equivalent comment to the comment for L1's detchar injections hveto safety analysis on page 35675
I ran hveto_safety on the injections mentioned above, looking for coincidences within 0.1sec time window.
The results of the analysis using >6 SNR omicron triggers can be found here: https://ldas-jobs.ligo-wa.caltech.edu/~tabbott/HvetoSafety/H1/O2/safetyinjections/D20170826/results/H1-HVETO_omicron_omicron-1187823990-125/safety_6.html
The configuration for the analysis can be found here: https://ldas-jobs.ligo-wa.caltech.edu/~tabbott/HvetoSafety/H1/O2/safetyinjections/D20170826/results/H1-HVETO_omicron_omicron-1187823990-125/H1-HVETO_CONF-1187823990-125.txt
I missed that there we 4 extra sets of injections, so I've redone the hveto safety analysis on the complete set of 180 injections.