Attached plot, from the lockloss tool, is zoomed in to the OMC-DCPD_SUM_OUT_DQ channel.
The plots shows that all of the other channels glitched after the OMC DC channel.
The channels were chosen by the lockloss tool, so maybe not a complete story yet.
Lock ended at 9:31UTC, and this plot shows 8.5 hours, which all but the very beginning of the lock.
Plots attached show that POP_A_LF_OUT is oscillating between 16200 and 17000 (800 counts) in the last 10 minutes of the previous lock, and is oscillating between 16200 and 16600 (400 counts) in the current lock (channel 1).
PO_A_LF_RF9_Q_ERR was oscillating between 35000 and 55000 (20000 counts), and is now between 55000 and 65000 (10000 counts, channel 2).
Lock loss at about 9:30UTC, and while nothing is broken, many small issues to relaocking.
Bottom Line - I can see the useism effecting the locks; arms in green, arms in red, PRMI, and DRMI, and that's made relocking twitchy.
Details:
- 3 or more times, I had the the PRMI locked and aligned, and when I brought SRM back in, it didn't just break the PRMI, the SRM watchdog tripped
- at least two times, HAM5 ISI also tripped
- during one of those, the BSC2 ISI tripped
- at least 3 times the DRMI locked but not good enough for WFS, and broke before I could manually fix it
- once DRMI locked and WFS did engage, I let it sit for a while, as I took a much needed break, if you must know, and even then it broke on it's way to Engage ASC
- DRMI just relocked and at 11:21UTC, it looks like the IFO might make it back to Low Noise
- at 11:25UTC, IFO is in Engage ASC and looks OK
Looking at ER8 and O1 data with the long transient search pipeline stampas we have noticed an excess of triggers corresponding to 2 narrow frequency lines at 39Hz and 43 Hz (plot 1). This excess of triggers happen at the beginning of H1 relock periods and is so large that it is visible in the spectrogram of h(t) (plot2). After a dozen of minutes, the excess of loud triggers disappear along with the lines. We are seen this problem since the beginning of ER8 but actually it seems that since October 8th, the excitation is smaller at each relock. Plot 3 shows that the long transient search is dominated by these triggers. We will develop a veto against these triggers, but we would be happy to see the pb fixed in the data!
TITLE: "10/08 [EVE Shift]: 23:00-07:00UTC (16:00-00:00 PDT), all times posted in UTC"
STATE Of H1: Observing at ~75 Mpc for the past 5 hours
SUPPORT: Sheila, Jeff K., Kiwamu
SHIFT SUMMARY: Few hiccups during lock acquisition. Sheila found that the CSOFT_Y was running away during DC_READOUT_TRANSITION (I believe) which probably caused a lockloss at DC_READOUT we had earlier. later ISS 2nd loop wouldn't engage. Kiwamu had to do it by hand. Microseism slowly coming down. Seismic activity in EQ band is nominal. Wind ~5mph.
INCOMING OPERATOR: Cheryl
ACTIVITY LOG:
00:51 Locked at NOMINAL_LOW_NOISE
01:00 Robert and Vinney to water pump area to do tamper injection.
01:45 Robert back
01:47 Switched to Undisturbed.
John, Gerardo
16:25 - Spun up LD, Turbo and QDP80
16:50 - LD was Calibrated with cal-leak, checked out.
16:55 - Valved in Turbo to vacuum volume, LD reported He background at 3.2 x 10-8 torr*L/sec, left system pumping.
17:08 - left Y-mid.
18:04 - Returned to Y-Mid to bag the butt welds of cylinder spool roll, 2 of them as half sections, for a total of 4 sections to test. Also continued to watch LD report..
19:20 - left Y-mid, LD reported background of 0.8 x 10-8 torr*L/sec.
20:25 - We returned to Y-Mid, LD reported a background of 5.3 x 10-9 torr*L/sec. Using a rate of 1 l/s of He, for 60 seconds dwell, leak hunting started:
* 3 butt welds of cylinder spool roll were tested, starting closer to the GV-11, no change on background for first 2 welds. Moved to the 3rd one, there was a signal, slow steady creep. Up until background reached 0.8 x 10-8 torr*L/sec.
* We paused for some time, gave the background a chance to drop with the VEA doors opened.
* We moved on to GV-11, since it is near to the butt welds, we checked its conflat and bellows with no change on background, other components had been checked before.
* Then some of the conflats on the spool piece by GV-10 were also tested, and again no change on the background.
* We moved to the turbo pump ports and turbo pump gate valve, they were tested with no change on the background.
* We returned to the butt weld, the 4th section (second butt weld) and tested it, we had a response and the background went up to 0.8 x 10-7 torr*L/sec, the signal was slow to creep up and it took sometime to show, thus creating a bit of confusion.
* We stopped leak testing because the background was too high, and it is going to take some time for the background to go down. We valved out the turbo pump from the vacuum volume.
We will continue with leak testing in the following days.
Note: While seen the background go up the turbo pump was isolated 3 times from the vacuum volume via its gate valve, and every time the background signal went down to <10-11 torr*L/sec
We are observing again at ~70 Mpc. Kiwamu had to engage ISS second loop by hand (alog 22449). It is unclear to me if we need to do this by hand after Maintenance day tomorrow. POP_A_LF fluctuates a lot and doens't look happy. Same goes for the control signals (they look unrested on the FOM, if the scale hasn't been changed). This is probably due to high microseism. Wind ~10 mph. Seismic activity in earthquake band looks nominal. Four ETMY saturations since we locked at Nominal Low Noise ~2 hours ago.
I noticed RF45 jumped above the reference at 0-10Hz just now while writing this log. Looks like it's been glitching for a while judging from the DMT Omega but didn't cause ETMY saturation so I didn't look up until now.
Made tamper injections at EY while LLO was having problems 6 to 6:30 local
Nutsinee, Sheila
We ran a2L again, here are the beam position results in meters (horizontal, then vertical):
Sheila, Nutsinee, Kiwamu,
For some reason, the 2nd loop ISS was not able to converge the PID loop at all during the engagement step. In the end, we engaged it by hand.
We will take a close look tomorrow during the maintenance period, but for the night, we will leave it as it is without any fix.
Instruction for manual engagement
Here is an instruction for the operators if the ISS runs into a difficulty in engaging the 2nd loop:
Software seem to be unchanged
Looking at the guardian and foton file, we confirmed that there was no change since Sep 9th. So we think this is not due to a softaware change or some sort. It could be some kind of hardware issues, but because of the fact that it runs OK once engaged, probably the hardware is not completely broken either.
We have lost lock twice in a row after turning up the gain of the soft loops, which seemed to cause an oscillation at around 40 mHz. We confirmed this a few times by tunring them on and off, and watching the oscillation come and go. We have now turned them on one at a time after increasing the power, and things seem OK. The attached screenshot shows the POP power fluctations durring the locks over the last 5 days. You can see that as the ground motion became large, the control signals from the soft loops become large and so do the fluctuations of the POP power.
CHARD yaw also doesn't seem to have enough juice to suppress the error signal with this kind of ground motion.
J. Kissel for the people actually doing the work: S. Dwyer, K. Izumi, N. Kijbunchoo, T. Schaffer A couple of problems, compounded with the high wind and microseism have made lock acquaisiton difficult today. Kiwmau and Shiela are still working on the last two I mention, and will likely aLOG details later: - The initial alignment was poor, as identified by a few lock losses that had too low a REFL power too soon so we ran through initial alignment. - During initial alignment, we found that the X-ARM's IR loop had too low a gain margin (see LHO aLOG 22443), for which a permanent solution will be tried tomorrow. - Once we finally got to DC READOUT, we found that all of the SOFT loops were difficult at best to close (that's the status now, again Shiela will aLOG more later) - Also, the ISS 2ND loop has become difficult to turn on (again, current status, Kiwamu's working on diagnosing) And again, on top of this the microseism has been at ~0.3 [um/s], and wind had gusted up to ~20 [mph] for a few hours in the early afternoon. Stay tuned.
TITLE: 10/12 [EVE Shift]: 23:00-07:00UTC (16:00-00:00 PDT), all times posted in UTC"
STATE Of H1: Lock Acquisition
OUTGOING OPERATOR: TJ
QUICK SUMMARY: We are still trying to lock. Have been having trouble with the SOFT loops (Sheila had to turn on -20dB filters multiple times). Now we are stalling at ENGAGE ISS 2ND LOOP.
J. Kissel, D. Macleod ECR E1500386 WP 5553 II 1126 I've begun the necessary front-end code infrastructure changes to copy over the hardware injection infrastructure on to the h1calex (PCAL X) front-end model. I've copied over the /opt/rtcds/userapps/release/cal/common/models/CAL_INJ_MASTER.mdl into the top level model, /opt/rtcds/userapps/release/cal/h1/models/h1calex.mdl and named the block "PINJX" to obey a similar naming convention as the X end model (in the event we eventually copy the infrastructure over to the Y-end as well). That means any X-end, photon calibrator injection channels will have the moniker of "PINJX" instead of the "standard" DARM injection channels that have "INJ," e.g. DARM Injection Channel Equivalent PCALX Injection Channel H1:CAL-INJ_TRANSIENT_EXC H1:CAL-PINJX_TRANSIENT_EXC H1:CAL-INJ_BLIND_OUT_DQ H1:CAL-PINJX_BLIND_OUT_DQ H1:CAL-INJ_ODC_HARDWARE_CTRL_EQ H1:CAL-PINJX_ODC_HARDWARE_CTRL_EQ etc. I've also cleaned up the top level a little bit with busses and tags, and connected the ODCCHAN output of the CAL-PINJX block to the pre-existing SHMEM IPC block that ships the ODC vector over to the h1iscex model to be collated and shipped to the ODC MASTER model in the corner station. In this way no new IPC was needed. However, the ODC channel for EX (which was essentially a dead, unused channel structure, with prefixes like H1:CAL-EX_ODC_...) will now have inputs like the INJ block in the corner station, as shown in one of the examples from above. Also note that, because we elected to *copy* the infrastructure instead of *move* the infrastructre, there will be 2 more 16 [kHz] channels in the science frames, and 2 more 16 [kHz] channels in the commissioning frames (plus the ODC channel stored at 256 [Hz]): H1:CAL-PINJX_HARDWARE_OUT_DQ* -- the sum of all requested (unblinded) hardware injections H1:CAL-PINJX_BLIND_OUT_DQ* -- the blind injection request H1:CAL-PINJX_TRANSIENT_OUT_DQ -- the transient injection requested ouput (prior to the sum before HARDWARE bank) H1:CAL-PINJX_CW_OUT_DQ -- the continuous wave requested output (prior to the sum before HARDWARE bank) H1:CAL-PINJX_ODC_CHANNEL_OUT_DQ* uint32 256 -- the new name of the ODC channel Finally, I've successful compiled the infrastructure and committed it to the userapps repo. I'll be working with Jim Batch to get the model installed first thing tomorrow morning, work with T.J. and Duncan to update the MEDM screen and fill out the ODC vector, Copy over the inverse actuation filter that Sudarshan has designed, then work with Eric, Peter, and Chris to update the various injection codes to absorb / handle the channel name changes. Stay tuned!
Title: 10/12 Day Shift 15:00-23:00 UTC (8:00-16:00 PST). All times in UTC.
State of H1: Lock Acquisition
Shift Summary: Locked until 18:12, struggled to relock after that. Sheila and I have been trying different things since. It just lost lock at DC_READOUT as I'm typing this.
Incoming operator: Nutsinee
Activity log:
TJ, Sheila, Keita
The LSC loop we use for locking the IR laser to the X arm is not great, (it doesn't have much gain margin which causes problems locking with different alignments.)
Nutsinee, Travis, and TJ have all run into trouble with the input aling step of initial alingment, where they could keep the arm locked on IR but the WFS would misalign the beam. Today we looked at the signals, saw that they were oscillating at around 126 Hz for alignments that increased the transmitted power. This was because the length loop was oscillating. We were able to temporarily fix this by reducing the gain in the loop from 0.1 to 0.05. We should take some time to reshape this loop.
Sheila, Jeff K, TJ
With the microseism still high and the winds not increasing, we decided to keep the 45mHz blends. We couldn't make it past DARM_WFS so we decided on an initial alignment, but now we can't even make it past the first step here. Sheila is working her magic so hopefully the next update will be more optimistic.
Here are the observed H1-L1 coherences as of October 12, 2015, as calculated by stochmon: https://ldas-jobs.ligo.caltech.edu/~thomas.callister/stochmonO1/stochmon.html Some plots attached. 100 mHz resolution. 1E-5 is the expected level of coherence due to uncorrelated noise. Frequency, coherence 33.6 Hz, 1E-3 34.6 Hz, 2E-2 35.4 Hz, 1E-3 42.5 Hz, 6E-4 (barely above background, could be a noise fluctuation) 64.0 Hz, 2E-4 (barely above background, could be a noise fluctuation) 74.0 Hz, 5E-4 (barely above background, could be a noise fluctuation) 128.0 Hz, 1E-2 256.0 Hz, 2E-3 331.8 Hz, 3E-4 (barely above background, could be a noise fluctuation) 351.1 Hz, 4E-4 (barely above background, could be a noise fluctuation) 559.9 Hz, 2E-4 (barely above background, could be a noise fluctuation) 736.0 Hz, 3E-3 (23rd harmonic of 32 Hz) 1904.0 Hz, 2E-3 (119th harmonic of 16 Hz) 2160.0 Hz, 4E-4 (135th harmonnic of 16 Hz) 2376.0 Hz, 4E-3 (297th harmonic of 8 Hz) 2768.0 Hz, 7E-3 (173rd harmonic of 16 Hz) 2984.0 Hz, 7E-4 (373rd harmonic of 8 Hz) 3001.2 Hz, 1E-1 (calibration lines) 4928.0 Hz, 3E-3 (77th harmonic of 64 Hz) 5016.0 Hz, 3E-3 (627th harmonic of 8 Hz) 5144.0 Hz, 2E-1 (largest coherence, 643rd harmonic of 8 Hz) 5616.0 Hz, 2E-3 (351st harmonic of 16 Hz) 5624.1 Hz, 6E-3 0.25 Hz, 4E-6 is the expected level of coherence due to uncorrelated noise. Frequency, coherence 34.25 to 35.25 Hz, 1E-3 36 to 37 Hz, 1.5E-3 37.25 Hz, 2E-3 42.5 Hz, 2E-4 128.0 Hz, 3E-3 256 Hz, 6E-4 1904.0 Hz, 4E-4 (119th harmonic of 16 Hz) 3552.25 Hz, 2E-4 4163.75 Hz, 5E-3 Hz 4576.0 Hz, 2E-4 (143rd harmonic of 32 Hz) 5016.0 Hz, 7E-4 (627th harmonic of 8 Hz) 5144.0 Hz, 1E-1 (largest coherence, 643rd harmonic of 8 Hz) 5584.5 Hz, 2E-2 5655.5 Hz, 1E-3
Continues the study on the H1-L1 h(t) coherence to date for O1. 0.001 Hz resolution, 2E-3 is the expected level of coherence due to uncorrelated noise. 0.5 Hz harmonics are seen in the H1-L1 coherence, averaged over all of O1 to data. See figures. Various other lines to be seen as well. Frequency, Coherence 18.50 Hz, 3E-2 19.00 Hz, 3E-2 19.50 Hz, 4E-2 20.00 Hz, 3.5E-2 23.50 Hz, 2.5E-2 25.00 Hz, 3E-2 25.50 Hz, 2.5E-2 32.00 Hz, 2E-2 36.7 Hz, 3E-2 42.5 Hz, 3E-2 43.0 Hz, 5E-2 48.0 Hz, 4E-2 61.243 Hz, 2E-2 71.0 Hz, 7E-2 71.50 Hz, 3E-2 100.00 Hz, 7E-2 128.00 Hz, 5E-1 256.00 Hz, 2E-1 896.0 Hz, 3.5E-2 952.0 Hz, 7E-2 1080.0 Hz, 1.5E-1