H1 dropped unexpectadly.  StripTools showed oscillations on LS-POP_A_LF and the SRM & PRM a few minutes before the lockloss.  Then Vinny noticed the increase in the EQ band (0.03-0.1Hz).  Then Terramon finally displayed the 5.6 EQ from Russia.  We dropped out before the R-wave estimate, but fairly certain the EQ is what knocked us out (and also knocked L1 down 3min after us).  

TITLE:  10/13 EVE Shift:  23:00-07:00UTC (16:00-00:00PDT), all times posted in UTC   

H1 is humming along at 77Mpc (handful of ETMy saturations).  useism continues its downward trend.  Wind is under 10mph and all looks quiet.  

And if you would like to get more realtime info, one can always take a look at the Control Room Screen Snapshots which are updated every minutes.

John, Gerardo

Today I got a chance to check the AIP that was installed last week (here), unfortunately the pump was railed, and (2x)unfortunately we don't know when it railed, no data available for this ion pump yet (working on it, data is comming soon).

We have an old valve that appears to be faulty, to patch that problem I added another valve in-line until we get a replacement, I used the aux cart to pump down the annulus system for 3 hours during the maintenance period, but that amount of time was not enough to keep the controller from railing.  We left the aux cart pumping on the annulus system, and it will remain pumping until the controller can maintain the system without railing.

16:27 - aux cart is connected and turned on, and remains on.
20:18 - aux cart was set on top of shipping foam, 3 points for seismic isolation.

Location of the aux cart:
It is on the East side of BSC5, see photo.

Related: alog 22199

We measured the output of the Low Voltage ESD monitor (D1500129) at the AA side at the remote rack using SR785 right after the IFO was locked up again after the maintenance.

No surpise was found, see attached. 'UR', 'UL' etc. are the analog measurements of the corresponding ESD monitor. Noise floor was measured with the inputs of SR785 short circuited.

Don't worry about LR trace, it looks as if it's noisier than the others but it's not, it's just the spectral leakage artefact (somehow I did not save the small BW measurement for LR).

'digital LL signal projected' is H1:SUS-ETMY_L3_MASTER_OUT_LL, but with a correction of 20/2^17 Volt/count for DAC conversion, zpk([50;50;3250], [2.2;2.2;152]) for SUS awhite type filters, and 2 *4.99/(1.2+15+4.99)*zpk([],[40]) for the monitor circuit.

The measurements agree well with the projection up to 1kHz. Higher than that, the SR785 noise dominates, but this should be good enough for our purpose.

The figure shows the features produced by the optic mount at the top of the periscope before maintenance (red) and 3 hours after recovery from maintenance (blue).  The amplitude seems reduced by nearly 10, but lets see how it ages. I used a watercolor paint brush to apply 5 minute epoxy to the base of the U-200 mount at the top of the periscope. I used one brush load. I was only able to paint the epoxy at the left and right sides of the mount because the middle was over a cutout in the base plate. After the epoxy was mostly cured, I adjusted weights on the periscope and optic, minimizing the peaks in IMC_F by placing them in valleys between periscope resonances.

Robert, Rick

Jordan, Sheila

In the summary pages, we can see that something non stationary appeared in DARM from about 80-250 Hz durring the long lock that spanned Oct 11th to 12th, and has stayed around.   Links to the spectra from the 11th  and the 12th.  

HVETO also came up with a lot of glitches in this frequency span starting on the 12th, (here) which were not around before.  These glitches are vetoed by things that seem like they could all be realted to corner statin ground motion: refl, IMC and AS WFS, all kinds of corner station seismic sensors, PEM accelerometers, MC suspensions.  

Although this noise seems to have appeared durring a time when the microseism was high for us, I think it is not directly related. (high microseism started approximately on the 9th, 2 days before this noise appeared and things are quieting down now but we still have the non stationary noise sometimes up to 200 Hz.)  

The blend switching could also seem like a culprit, but the blends were not switched at the begining of the lock in which this noise appeared, and we have been back on the normal (90mHz blends) today but we still have this noise.  We've seen scattering from the OMC with velocities this high before (17264 and 19195).  

Nutsinee and Robert have found that some of the glitches we are having today are due to RF45, but this doesn't seem to be the case on the 12th. 

I ran Dan's code to compare stationarity between two hours on Oct 5th and Oct 12th.  The closure of the beam diverter on the 6th was thought to help stationarity in the 70-100hz region, so I included versions zoomed in on that frequency range.

I tried to pick times that were not affected by huge glitches / range drops. On the 5th the two hours were 1:00-3:00UTC and on the 12th 00:00-02:00UTC.

plots attached

TITLE:  10/13 EVE Shift:  23:00-07:00UTC (16:00-00:00PDT), all times posted in UTC     

STATE of H1:  In Observation Mode at 75Mpc

Outgoing Operator:  Jim

Support:  Occupied Control Room, Usual Suspects

Quick Summary:

Jim's computer crashed so he lost his log of activities for his shift.  He did not have much to pass on other than being out for ~8hrs for Maintenance Day and recently getting H1 back up to Observation Mode.  Off-topic, but he also showed me how to transition the ISI Stage-1 Blend Filters for high useism conditions (there's an actual sticky note on the Ops Work station about this.).

Richard, Jeff, Evan

It had been on for at least several months.

Darkhan T., Travis S.

During today's maintenance period, we took PCal calibration measurements for both end stations.  Attached are photos of the datasheets (mostly for our own reference).  Analysis of the data to follow.

Added 205ml cooling water to the TCSX chiller. First water added to TCSX in the past month.

Added 600ml cooling water to the TCSY chiller. Given the amount of water loss with the TCSY Chiller but not with the TCSX Chiller over the past three weeks, I am suspicious there is a leak in the TCSY cooling loop.    

I added marks (at 1/4 , 1/2, and 3/4) next to the site glass so we can more easily track water loss.   

and has been continually since Monday afternoon at ~39hz (1150rpm [19hz] at motor/pump.)  Let me know if this is a problem.

I have disabled the IMC WFS offsets that Keita and Cheryl wanted to remove (alog 22370).

This may help the recent ISS issue (alog 22449) somewhat, but probably not a lot.

Background

I was investigating why the ISS recently started having the trouble in engaging the 2nd loop. A first thing I noticed is relatively high RIN after the IMC. It seems that the IMC adds some extra RIN below 1 Hz as the light goes through it. A large RIN at low frequencies can easily hinder the PID control of the 2nd loop when engaging. So this seemed suspicious to me.

Since the ISS 2nd loop had been running without a significant issue in the past, perhaps until the recent change on the IMC WFS (alog 22362), I speculated that the current IMC alignment is at a non-optimal point where the low frequency RIN became worse. One factor which affects the IMC alignment is the WFS offsets that are meant to minimize the jitter-to-RIN couplings. In the end, I decided to get rid of the offsets because it gave a better RIN coupling.

IMC_LOCK guardian edited

To disable the offset, I have commented out lines 555 and 556 in the IMC_LOCK guardian. They are now,

#ezca.switch('IMC-DOF_2_P', 'OFFSET', 'ON')
#ezca.switch('IMC-DOF_1_Y', 'OFFSET', 'ON')
 

RIN is better without the offsets

I did a simple test where I measured RIN after the IMC with and without the WFS offsets. Note that the offsets have been applied only on DOF2_P and DOF1_Y. They have not yet been optimized after the change in the IMC WFS. The measured RIN are shown in the attached screenshot.

Blue: RIN at the ISS in-vac array without the offsets, Cyan: RIN at the ISS in-vac array with the offsets, Brown and red: RIN before the IMC with and without the offsets, respectively. The 1st loop was always ON while the 2nd loop was always OFF. The input power was about 2 W.

First of all, it is clear that RIN after the IMC is higher than that before the IMC by a factor of more than 10 at around 0.1 Hz. I am not sure how long it has been like this.

Next, as shown in the plot, removing the offsets reduced the excess RIN below 1 Hz somewhat at the in-vac ISS array. This was repeatable and therefore I think the improvement is real. Additionally, it got rid of a nasty structure at around 600 Hz.  Also, some peaks above 100 Hz decreased their peak heights by a factor of few. These improvements were consistenly observed at IMC-MC2_TRANS_SUM as well. Therefore I think they are real improvement in RIN and not some kind of spurious couplings such as spatial jitter.

So overall, the configuration without the offsets seems better in terms of RIN. Perhaps this is an indication of better WFS sensing. I decided to disable the offsets in the IMC_LOCK guardian. I did not try to optimize the offsets yet. So this can be a next action item.

I've rearranged the PRMI branch in the DRMI guardian, and added states to the ISC_LOCK guardian to request both these states and the ALIGN_IFO states used for PRMI  (finshing unfinished item from WP5533).  This is for use when you think that the DRMI alignment is bad (based on the AS camera and the flashes as seen in dataviewer)  and will not lock.  If DRMI is slow to lock but has been locked recently and looks well aligned on the AS camera, this procedure is probably not your best bet.

We will test this durring maintence recovery today, but the instructions replacing those in 21745 will be:

• From ISC_LOCK, request LOCK_PRMI.  
• Once it locks, align BS and PRM by hand until you are satisfied with the alingment (POP18_I_NORM_MON around 50 counts and POP90_I_NORM_MON around 80 counts, fairly stable)
• Request DRMI_LOCKED.  The guardian will realign SRM, attempt to make the transition without unlocking DRMI, and go back to LOCK_DRMI_1F if it fails. 

done!

On Tuesday, while most of the IFO was down for various maintenance tasks, I grabbed a bunch of SUS spectra and started a Template directory. 

I set all of the references to be the ~quiet, unlocked data from the down period of the IFO.  These spectra could be run when trying to troubleshoot locking.  On Tuesday, I was only able to collect the following, located in /ligo/home/ops/Templates/SUS_Spectra/

BS_NOISEMON_spectra.xml

BS_OSEMINF_spectra.xml      

PR2_NOISEMON_spectra.xml

PR2_OSEMINF_spectra.xml  

PRM_NOISEMON_spectra.xml

PRM_OSEMINF_spectra.xm

SR3_NOISEMON_spectra.xml

SR3_OSEMINF_spectra.xml

SR2_NOISEMON_spectra.xml

SR2_OSEMINF_spectra.xml

SRM_NOISEMON_spectra.xml

SRM_OSEMINF_spectra.xml

ITMX_NOISEMON_spectra.xml

ITMX_OSEMINF_spectra.xml

ITMY_NOISEMON_spectra.xml

ITMY_OSEMINF_spectra.xml
 

NOTE - Many of the lower stage NOISEMON channels look... weird.  They should be used as a before/.after snapshot for troubleshooting only.  Work to improve what the various strangenesses of these channels is on a few low priority to-do lists.

There are templates ready to collect the balance of the SUS spectra data, namely ETMs, MCs, PR3.  We should do this when the IFO is set to DOWN for whatever reason.  The spectra are quick to run, then update all references (currently bogus data) and resave.

We set all quadrant gains of IMC WFSA to [1 1 1 1] from [1, 0.25, 1, 4]. (WFSB was already all 1.)

We also disabled IMCWFS error offsets in servo filters.

After this, we steered IM2 to bring the beam position on IM4 trans back (at first we tried IM3, but it would make the OSEM output to become larger, and they're already close to saturation).

IFO locked after this without any problem.

We haven't done any jitter coupling optimization and I don't know if Robert had time to do it.