Lock loss at about 9:30UTC, and while nothing is broken, many small issues to relaocking.

Bottom Line - I can see the useism effecting the locks; arms in green, arms in red, PRMI, and DRMI, and that's made relocking twitchy.

Details:

- 3 or more times, I had the the PRMI locked and aligned, and when I brought SRM back in, it didn't just break the PRMI, the SRM watchdog tripped

- at least two times, HAM5 ISI also tripped

- during one of those, the BSC2 ISI tripped

- at least 3 times the DRMI locked but not good enough for WFS, and broke before I could manually fix it

- once DRMI locked and WFS did engage, I let it sit for a while, as I took a much needed break, if you must know, and even then it broke on it's way to Engage ASC

- DRMI just relocked and at 11:21UTC, it looks like the IFO might make it back to Low Noise

- at 11:25UTC, IFO is in Engage ASC and looks OK

Looking at ER8 and O1 data with the long transient search pipeline stampas we have noticed an excess of triggers corresponding to 2 narrow frequency lines at 39Hz and 43 Hz (plot 1). This excess of triggers happen at the beginning of H1 relock periods and is so large that it is visible in the spectrogram of h(t) (plot2). After a dozen of minutes, the excess of loud triggers disappear along with the lines. We are seen this problem since the beginning of ER8 but actually it seems that since October 8th, the excitation is smaller at each relock. Plot 3 shows that the long transient search is dominated by these triggers. We will develop a veto against these triggers, but we would be happy to see the pb fixed in the data!

TITLE: "10/08 [EVE Shift]: 23:00-07:00UTC (16:00-00:00 PDT), all times posted in UTC"

STATE Of H1: Observing at ~75 Mpc for the past 5 hours

SUPPORT: Sheila, Jeff K., Kiwamu

SHIFT SUMMARY: Few hiccups during lock acquisition. Sheila found that the CSOFT_Y was running away during DC_READOUT_TRANSITION (I believe) which probably caused a lockloss at DC_READOUT we had earlier. later ISS 2nd loop wouldn't engage. Kiwamu had to do it by hand. Microseism slowly coming down. Seismic activity in EQ band is nominal. Wind ~5mph.

INCOMING OPERATOR: Cheryl

ACTIVITY LOG:

00:51 Locked at NOMINAL_LOW_NOISE

01:00 Robert and Vinney to water pump area to do tamper injection.

01:45 Robert back

01:47 Switched to Undisturbed.

John, Gerardo

16:25 - Spun up LD, Turbo and QDP80
16:50 - LD was Calibrated with cal-leak, checked out.
16:55 - Valved in Turbo to vacuum volume, LD reported He background at 3.2 x 10-8 torr*L/sec, left system pumping.
17:08 - left Y-mid.
18:04 - Returned to Y-Mid to bag the butt welds of cylinder spool roll, 2 of them as half sections, for a total of 4 sections to test.  Also continued to watch LD report..
19:20 - left Y-mid, LD reported background of 0.8 x 10-8 torr*L/sec.
20:25 - We returned to Y-Mid, LD reported a background of 5.3 x 10-9 torr*L/sec.  Using a rate of 1 l/s of He, for 60 seconds dwell, leak hunting started:
    * 3 butt welds of cylinder spool roll were tested, starting closer to the GV-11, no change on background for first 2 welds.  Moved to the 3rd one, there was a signal, slow steady creep. Up until background reached 0.8  x 10-8 torr*L/sec.
    * We paused for some time, gave the background a chance to drop with the VEA doors opened.
    * We moved on to GV-11, since it is near to the butt welds, we checked its conflat and bellows with no change on background, other components had been checked before.
    * Then some of the conflats on the spool piece by GV-10 were also tested, and again no change on the background.
    * We moved to the turbo pump ports and turbo pump gate valve, they were tested with no change on the background.
    * We returned to the butt weld, the 4th section (second butt weld) and tested it, we had a response and the background went up to  0.8 x 10-7 torr*L/sec, the signal was slow to creep up and it took sometime to show, thus creating a bit of confusion.
    * We stopped leak testing because the background was too high, and it is going to take some time for the background to go down.  We valved out the turbo pump from the vacuum volume.

We will continue with leak testing in the following days.


Note: While seen the background go up the turbo pump was isolated 3 times from the vacuum volume via its gate valve, and every time the background signal went down to <10-11 torr*L/sec

We are observing again at ~70 Mpc. Kiwamu had to engage ISS second loop by hand (alog 22449). It is unclear to me if we need to do this by hand after Maintenance day tomorrow. POP_A_LF fluctuates a lot and doens't look happy. Same goes for the control signals (they look unrested on the FOM, if the scale hasn't been changed). This is probably due to high microseism. Wind ~10 mph. Seismic activity in earthquake band looks nominal. Four ETMY saturations since we locked at Nominal Low Noise ~2 hours ago.

I noticed RF45 jumped above the reference at 0-10Hz just now while writing this log. Looks like it's been glitching for a while judging from the DMT Omega but didn't cause ETMY saturation so I didn't look up until now.

Made tamper injections at EY while LLO was having problems 6 to 6:30 local

Nutsinee, Sheila

We ran a2L again, here are the beam position results in meters (horizontal, then vertical):

Sheila, Nutsinee, Kiwamu,

For some reason, the 2nd loop ISS was not able to converge the PID loop at all during the engagement step. In the end, we engaged it by hand.

We will take a close look tomorrow during the maintenance period, but for the night, we will leave it as it is without any fix.

Instruction for manual engagement

Here is an instruction for the operators if the ISS runs into  a difficulty in engaging the 2nd loop:

• If ISS does not engage the 2nd loop for more than 5 minutes in the "ENGAGE_ISS_2ND_LOOP" state, please proceed to the following steps.
• Check the IMC guardian state.
  • If it has fallen back to 'LOCKED' or something equivalent, re-request ''ISS_ON" manually.
  • Make sure that the IMC_LOCK guardian is in the "PREPARE_ISS" state.
• Open up the medm window for the 2nd loop (SITEMAP --> PSL ---> ISS --> SECOND LOOP).
  • Getting to the 2nd loop medm from the generic ISS medm can be tricky for some people, so see the 1st attachment to check where to click.
• In the 2nd loop medm, check how the output is converging. See the 2nd attached.
  • If the output value is bigger than +/- 5 and does not move, please call an expert.
  • If the output value is moving (or often oscillating), then you can manually turn on the loop by pressing a green "ON" button at the top as shown in the 2nd attached screenshot. A rule of thomb is to click the "ON" button when the output value is close to zero.
• After the manual engagement, make sure that the output value becomes smaller, ideally close to zero without any oscillations.
  • Then the guardian will wait for 60 seconds and automatically move onto the "CLOSE_ISS" state. We are done. It is going to automatically take you to the "ISS_ON".
• If the above does not work or break the lock, please call an expert.

Software seem to be unchanged

Looking at the guardian and foton file, we confirmed that there was no change since Sep 9th. So we think this is not due to a softaware change or some sort. It could be some kind of hardware issues, but because of the fact that it runs OK once engaged, probably the hardware is not completely broken either.

We have lost lock twice in a row after turning up the gain of the soft loops, which seemed to cause an oscillation at around 40 mHz. We confirmed this a few times by tunring them on and off, and watching the oscillation come and go.  We have now turned them on one at a time after increasing the power, and things seem OK.  The attached screenshot shows the POP power fluctations durring the locks over the last 5 days. You can see that as the ground motion became large, the control signals from the soft loops become large and so do the  fluctuations of the POP power.  

CHARD yaw also doesn't seem to have enough juice to suppress the error signal with this kind of ground motion.  

J. Kissel for the people actually doing the work: 
S. Dwyer, K. Izumi, N. Kijbunchoo, T. Schaffer

A couple of problems, compounded with the high wind and microseism have made lock acquaisiton difficult today. Kiwmau and Shiela are still working on the last two I mention, and will likely aLOG details later:
- The initial alignment was poor, as identified by a few lock losses that had too low a REFL power too soon so we ran through initial alignment.
- During initial alignment, we found that the X-ARM's IR loop had too low a gain margin (see LHO aLOG 22443), for which a permanent solution will be tried tomorrow.
- Once we finally got to DC READOUT, we found that all of the SOFT loops were difficult at best to close (that's the status now, again Shiela will aLOG more later)
- Also, the ISS 2ND loop has become difficult to turn on (again, current status, Kiwamu's working on diagnosing)

And again, on top of this the microseism has been at ~0.3 [um/s], and wind had gusted up to ~20 [mph] for a few hours in the early afternoon.

Stay tuned.

TITLE: 10/12 [EVE Shift]: 23:00-07:00UTC (16:00-00:00 PDT), all times posted in UTC"

STATE Of H1: Lock Acquisition

OUTGOING OPERATOR: TJ

QUICK SUMMARY: We are still trying to lock. Have been having trouble with the SOFT loops (Sheila had to turn on -20dB filters multiple times). Now we are stalling at ENGAGE ISS 2ND LOOP.

J. Kissel, D. Macleod
ECR E1500386
WP 5553
II 1126

I've begun the necessary front-end code infrastructure changes to copy over the hardware injection infrastructure on to the h1calex (PCAL X) front-end model. I've copied over the 
/opt/rtcds/userapps/release/cal/common/models/CAL_INJ_MASTER.mdl
into the top level model,

/opt/rtcds/userapps/release/cal/h1/models/h1calex.mdl
and named the block "PINJX" to obey a similar naming convention as the X end model (in the event we eventually copy the infrastructure over to the Y-end as well). That means any X-end, photon calibrator injection channels will have the moniker of "PINJX" instead of the "standard" DARM injection channels that have "INJ," e.g.

        DARM Injection Channel         Equivalent PCALX Injection Channel
   H1:CAL-INJ_TRANSIENT_EXC                H1:CAL-PINJX_TRANSIENT_EXC
   H1:CAL-INJ_BLIND_OUT_DQ                 H1:CAL-PINJX_BLIND_OUT_DQ
   H1:CAL-INJ_ODC_HARDWARE_CTRL_EQ         H1:CAL-PINJX_ODC_HARDWARE_CTRL_EQ

etc.

I've also cleaned up the top level a little bit with busses and tags, and connected the ODCCHAN output of the CAL-PINJX block to the pre-existing SHMEM IPC block that ships the ODC vector over to the h1iscex model to be collated and shipped to the ODC MASTER model in the corner station. In this way no new IPC was needed. However, the ODC channel for EX (which was essentially a dead, unused channel structure, with prefixes like H1:CAL-EX_ODC_...) will now have inputs like the INJ block in the corner station, as shown in one of the examples from above.

Also note that, because we elected to *copy* the infrastructure instead of *move* the infrastructre, there will be 2 more 16 [kHz] channels in the science frames, and 2 more 16 [kHz] channels in the commissioning frames (plus the ODC channel stored at 256 [Hz]):
H1:CAL-PINJX_HARDWARE_OUT_DQ*                 -- the sum of all requested (unblinded) hardware injections
H1:CAL-PINJX_BLIND_OUT_DQ*                    -- the blind injection request
H1:CAL-PINJX_TRANSIENT_OUT_DQ                 -- the transient injection requested ouput (prior to the sum before HARDWARE bank)
H1:CAL-PINJX_CW_OUT_DQ                        -- the continuous wave requested output (prior to the sum before HARDWARE bank)
H1:CAL-PINJX_ODC_CHANNEL_OUT_DQ* uint32 256   -- the new name of the ODC channel

Finally, I've successful compiled the infrastructure and committed it to the userapps repo. I'll be working with Jim Batch to get the model installed first thing tomorrow morning, work with T.J. and Duncan to update the MEDM screen and fill out the ODC vector, Copy over the inverse actuation filter that Sudarshan has designed, then work with Eric, Peter, and Chris to update the various injection codes to absorb / handle the channel name changes.

Stay tuned!

Title: 10/12 Day Shift 15:00-23:00 UTC (8:00-16:00 PST).  All times in UTC.

State of H1: Lock Acquisition

Shift Summary: Locked until 18:12, struggled to relock after that. Sheila and I have been trying different things since. It just lost lock at DC_READOUT as I'm typing this.

Incoming operator: Nutsinee

Activity log:

• 14:19 ETMY sat x2
• 14:55 Bubba bangging on control room
• 15:04 ETMY
• 15:07 ETMY
• 16:15 John, Gerardo to MY for leak checking
• 16:41 Jenne to MY to swap cars
• 16:59 Jenne back
• 16:59 Richard and HFD to MY
• 17:15 Gerardo, John back
• 17:18 Richard and HFD back
• 17:52 Keita and Elli to EY for LV ESD Noise measurement
• 18:04 Gerardo to MY
• 18:12 Lockloss
• 18:20 Blends switched to Quiet_90
• 18:55 Dick to Optics Lab
• 18:56 Blends switched back to 45mHz
• 19:12 Dick back
• 19:25 Gerardo back
• 19:36 Keita back
• 19:53 Starting Initial Alignment
• 20:17 John, Gerardo to MY
• 22:36 John, Gerardo back
• 22:42 Robert driving down Y arm to get equipment in beam tube enclosure between CS and MY

TJ, Sheila, Keita

The LSC loop we use for locking the IR laser to the X arm is not great, (it doesn't have much gain margin which causes problems locking with different alignments.)

Nutsinee, Travis, and TJ have all run into trouble with the input aling step of initial alingment, where they could keep the arm locked on IR but the WFS would misalign the beam.  Today we looked at the signals, saw that they were oscillating at around 126 Hz for alignments that increased the transmitted power.  This was because the length loop was oscillating.  We were able to temporarily fix this by reducing the gain in the loop from 0.1 to 0.05.  We should take some time to reshape this loop.  

Sheila, Jeff K, TJ

With the microseism still high and the winds not increasing, we decided to keep the 45mHz blends. We couldn't make it past DARM_WFS so we decided on an initial alignment, but now we can't even make it past the first step here. Sheila is working her magic so hopefully the next update will be more optimistic.

Here are the observed H1-L1 coherences as of October 12, 2015, as calculated by stochmon:
https://ldas-jobs.ligo.caltech.edu/~thomas.callister/stochmonO1/stochmon.html

Some plots attached.

100 mHz resolution. 1E-5 is the expected level of coherence due to uncorrelated noise.
Frequency, coherence
33.6 Hz, 1E-3
34.6 Hz, 2E-2
35.4 Hz, 1E-3
42.5 Hz, 6E-4 (barely above background, could be a noise fluctuation)
64.0 Hz, 2E-4 (barely above background, could be a noise fluctuation)
74.0 Hz, 5E-4 (barely above background, could be a noise fluctuation)
128.0 Hz, 1E-2
256.0 Hz, 2E-3
331.8 Hz, 3E-4 (barely above background, could be a noise fluctuation)
351.1 Hz, 4E-4 (barely above background, could be a noise fluctuation)
559.9 Hz, 2E-4 (barely above background, could be a noise fluctuation)
736.0 Hz, 3E-3 (23rd harmonic of 32 Hz)
1904.0 Hz, 2E-3 (119th harmonic of 16 Hz)
2160.0 Hz, 4E-4 (135th harmonnic of 16 Hz)
2376.0 Hz, 4E-3 (297th harmonic of 8 Hz)
2768.0 Hz, 7E-3 (173rd harmonic of 16 Hz)
2984.0 Hz, 7E-4 (373rd harmonic of 8 Hz)
3001.2 Hz, 1E-1 (calibration lines)
4928.0 Hz, 3E-3 (77th harmonic of 64 Hz)
5016.0 Hz, 3E-3 (627th harmonic of 8 Hz)
5144.0 Hz, 2E-1 (largest coherence, 643rd harmonic of 8 Hz)
5616.0 Hz, 2E-3 (351st harmonic of 16 Hz)
5624.1 Hz, 6E-3 


0.25 Hz, 4E-6 is the expected level of coherence due to uncorrelated noise.
Frequency, coherence
34.25 to 35.25 Hz, 1E-3
36 to 37 Hz, 1.5E-3
37.25 Hz, 2E-3
42.5 Hz, 2E-4
128.0 Hz, 3E-3
256 Hz, 6E-4
1904.0 Hz, 4E-4 (119th harmonic of 16 Hz)
3552.25 Hz, 2E-4
4163.75 Hz, 5E-3 Hz
4576.0 Hz, 2E-4 (143rd harmonic of 32 Hz)
5016.0 Hz, 7E-4 (627th harmonic of 8 Hz)
5144.0 Hz, 1E-1 (largest coherence, 643rd harmonic of 8 Hz)
5584.5 Hz, 2E-2
5655.5 Hz, 1E-3

J. Kissel, T. Schaffer, S. Dwyer

After loosing lock, "the plan" had been to switch back to 90 [mHz] blend fliters, after having switched to 45 [mHz] over the weekend (see LHO aLOG 22410). After loosing lock at 18:12 UTC, by ~18:30 UTC we had BSC platforms switched to 90 [mHz] blends. Sheila commented that motion is still too large, so we switched back to 45 [mHz] blends by 19:00 UTC, and we're resuming lock acquisition. We'll see how it goes -- the wind is on its way up...

I attach some relevant screenshots of the amplitude of motion over the past 12 hours.

Summary:

Last night I performed a number of hardware injections see aLog 22424 and aLog 22426.

Waveforms:

The waveforms were provided by Max and Francesco on behalf of the Burst group.

The single-column ASCII files are located in the injection SVN: https://daqsvn.ligo-la.caltech.edu/svn/injection/hwinj/Details/Inspiral/${IFO}/ringburstinj*.out

The parameters files are located: https://daqsvn.ligo-la.caltech.edu/svn/injection/hwinj/Details/Inspiral/ringburstinj*.xml.gz

First Set of Injections:

Originally the plan was to space the injections 180s apart.

I used tinj to schedule the injections so they are all H1L1 coherent injections.

The following injections were successful:
1128657857 1 1.0 ringburstinj1a_1126259456_
1128658217 1 1.0 ringburstinj2a_1126259456_
1128658577 1 1.0 ringburstinj3a_1126259456_
1128658937 1 1.0 ringburstinj4a_1126259456_
1128659297 1 1.0 ringburstinj5a_1126259456_

Note that the GPS time above (first column) is the time the injection began. The end times are ~6s after this time. The exact end time can be calculated from the parameter files.

Also recall from aLog 21487 that all analysis should have a 255us advance.

The following injections were skipped, ie. not injected, because tinj couldn't inject signals every 180s. The following were skipped injections and thus not successful:
1128658037 1 1.0 ringburstinj1b_1126259456_
1128658397 1 1.0 ringburstinj2b_1126259456_
1128658757 1 1.0 ringburstinj3b_1126259456_
1128659117 1 1.0 ringburstinj4b_1126259456_
1128659477 1 1.0 ringburstinj5b_1126259456_

We then had an external alert during the test. The injection in the queue was canceled at H1 but the injection was not canceled at L1. Duncan, Peter, Alex are investigating why this happened. The following injection was not performed at H1 but it was performed successfully at L1:
1128659657 1 1.0 ringburstinj6a_1126259456_

I then deleted the future injections from the schedule since it did not seem like L1 properly ignored the last injection. See aLog 22425 for the lines that were deleted.

Second Set of Injections:

We continued the hardware injections later in the night after waiting so the external alert had a stretch of clean data. The injections were spaced 360s apart this time and there was no problem with tinj handling injections this frequent.

The following injections were successful:
1128666017 1 1.0 ringburstinj1b_1126259456_
1128666377 1 1.0 ringburstinj2b_1126259456_
1128666737 1 1.0 ringburstinj3b_1126259456_
1128667097 1 1.0 ringburstinj4b_1126259456_
1128667457 1 1.0 ringburstinj5b_1126259456_
1128667817 1 1.0 ringburstinj6a_1126259456_
1128668177 1 1.0 ringburstinj6b_1126259456_
1128668537 1 1.0 ringburstinj7b_1126259456_
1128668897 1 1.0 ringburstinj8a_1126259456_
1128669257 1 1.0 ringburstinj7a_1126259456_
1128669617 1 1.0 ringburstinj8b_1126259456_
1128669977 1 1.0 ringburstinj9a_1126259456_
1128670337 1 1.0 ringburstinj10a_1126259456_
1128670697 1 1.0 ringburstinj11a_1126259456_
1128671057 1 1.0 ringburstinj12a_1126259456_
1128671417 1 1.0 ringburstinj13a_1126259456_
1128671777 1 1.0 ringburstinj14a_1126259456_
1128672497 1 1.0 ringburstinj15a_1126259456_

We just lost lock at 18:12 UTC