Shift Summary - Evening Transition (Maintenance Day)

TITLE:   Oct 6 DAY Shift 15:00-23:00UTC (08:00-04:00 PDT), all times posted in UTC

STATE Of H1: Aligning/Locking

LOCK DURATION: N/A

SUPPORT: Sheila, Jenne, TJ

INCOMING OPERATOR: Nutsinee

Activity log:

14:55 Safety Kleen is on site

14:57 Sheila performing scattered light noise hunting testing on EX WP#5528

14:58 Richard starting his temperature Beckhoff system room temperature sensor work. WP#5534

15:00 Jeff B running forklift WP#5506

15:08 Lockloss

15:12 Jody to begin executing WP#5537 upon completion of Jeff B’s work (#5506)

15:13 Ken out to EY for Solar Panel work WP#5493

15:15 Jim batch informed of DMT maintenance. EQ blrms, range, IR integrand. etc

15:19 Joe out to LVEA to check water, batteries. etc

15:22  Robert’s equipment is unloaded and he will begin PEM injections

15:24 Greg Mendell informed me of the start of the DMT update WP#5522

15:26 Joe turning on lights in LVEA

15:27 Kyle and Gerardo out to  execute WP#5532. Informed me that new LN2 vendor will be arriving on site today.

15:30 Richard is turning on wireless in LVEA. WE NEED TO REMIND THE SWEEP TEAM TO TURN IT OFF.

15:35  Switched observatory mode to Corrective Maintenance

15:40  NORCO on site for LN2 delivery Y end

15:42 Vinnie into the electronics room to investigate two accelerometers.

15:46 Jody called to inform that forklift was loaded and they’re headed down X arm. WP#5537

15:49 Richard finished temp sensors in corner. He’s headed to EX

15:51  Sheila will be breaking the current lock. Betsy will do some charge measurements.

15:53 Hugh out to do weekly HEPI fluid checks at all out buildings and corner.

16:02 Christina called. Leaving EY headed to EX.

16:06 John W driving down to EY to meet LN2 driver.

16:08 Jason in to LDR to do regular weekly Watchdog reset .

16:15 Septic truck on site

16:19 Richard called to confirm operation of sensors at EY

16:23 Ellie and Miguel headed out to ISCT6 to execute WP#5540. (camera)

16:26  Gerardo back from EX to remove some vac equipment to take back to EX

16:30 Richard and Fil at EX for Ecat Temp sensor install/test

16:40  Second Norco truck on site to service CP2

16:42  Christina leaving EX. Returning to get mid station key.

17:01 Sheila is going to start the A2L script testing.

17:05  Vinnie to EX

17:06  Jodie and Jeff done at MX and headed to MY.

17:07  Kyle back from EX

17:07  Safety Kleen leaving site

17:08 John Back from EY

17:15  Richard and Fil are done.

17:28 Eliie and Miguel out of the LVEA for the moment. They plan to return.

17:30  Fil out to Biergarten to label TCS racks.

17:37 Christina leaving MX and heading to MY

17:48  Ellie and Miguel back out to ISCT6. Misaligned SRM, PRM and ITMX for single beam

17:55 Jody called. They’re done at the Mid stations. They’re going to park the forklift by the LSB.

17:57  Richard down to EY to join Ken.

18:03 Ace finished on Y arm and heading down X arm

18:04  Bubba moving something with a forklift from LSB to OSB.

18:05 Vinnie called to let me know that he’s at EY.

18:14 Jody and Jeff out to LVEA to pull temperature data from 3IFO containers

18:19  Hugh to end stations for photos

18:20 Christina and Karen into LVEA for cleaning

18:23 EY LN2 truck leaving

18:26 Evan out to LVEA to look at ISC racks by the PSL

18:29 Evan back

18:44  Ellie and Miguel out of LVEA

18:46 Ellie back out to put in a beam dump

18:47 Jody and Jeff are out of the LVEA

18:48  Carlos done for the day

18:52  Ellie out

18:54 Landry into LVEA for photos

19:06 Richard out to LVEA to turn wireless router off and turn off power distribution cart by output arm.

19:15 Gerardo called from EX (phone showed MY). Pumpe replaced but auc cart neds to stay running for another hour to hour/half

19:30 Kyle leaving Xarm and travelling to EY to check something

19:31 Vinnie back from EY

19:38 Robert and Vinnie ou to EY to position a magnetometer

19:57 LVEA sweep team headed out to the LVEA

20:01 Science mode (after locking) will be hindered by an aux pump still running at EX, Dave and Jonathan doing computer work and an SDF change that’s going to happen due to PEM work at EY to investigate glitching. THe latter will require a work permit that will remain open as the present work will be a temporary fix.

20:18 Ellie is still in the LVEA for Fine Aligning of ISCT6 LVEA sweep will be final upon her exit. Wireless Router was turned back on for her use.

20:35 Dither align script has been renamed from what the wiki page says “ ./dither_align.py” to “./ditherAlign.py”

20:35  Begin initial alignment

22:25  Jeff B into the optics lab

22:33 Jeff B out of the optics lab

23:00 Still struggling to re-lock. Handing off to Nutsinee

 

Shift Summary: See Activity Log

Low Water Alarm on PSL Diode Chiller

   The PSL Diode was showing a low water alarm this afternoon. I added about a quart, and cleared the alarm.

Likely Bad Endevco Channels for CS Acc

Two accelerometers in the corner station have been showing unusual looking spectra.  During maintenance today I moved them to different endevco channels to see if that was the problem, and it their resulting spectra would suggest that it is.

 

H1:PEM-CS_ACC_LVEAFLOOR_BS_Z showed excess low frequency noise in its spectra.  When switched to a different endevco channel the noise was noticeably lower below ~6-7 Hz.  Both images are attached to this post.

 

H1:PEM-CS_ACC_LVEAFLOOR_HAM1_Z also showed an unusual spectrum.  When ran through a different endevco channel the overall noise level changed by a factor of ~10.  Two images of these spectra are attached as well.

Add Water to TSC-Y Chiller

   Added 250ml water to the TSC-Y chiller. This is the second time I've added water to this chiller. The TCS-X chiller water level was fine.   

X-END BSC5 Annulus Ion Pump

(Kyle, Gerardo)

Removed and replaced the AIP for BSC5.
Pump had failed early last week.

Temp/RH Data for SUS & 3IFO storage Cabinets

   Attached are the September temperature and relative humidity plots for the SUS and 3IFO controlled environment storage boxes. All RH data is good. There were a couple of temperature spikes on DB1 and DB4 but no issues with humidity. 

I have powered down DB2 and DB3, which are currently unused.

New Temperature Monitors in Electroncis Rooms

Richard et al hooked up the missing temperature probes in the electroncis rooms. Attached is the new medm screen.

Channel names

                H1:PEM-C_MSR_RACK1_TEMPERATURE
                H1:PEM-C_MSR_RACK2_TEMPERATURE
                H1:PEM-C_CER_RACK1_TEMPERATURE
                H1:PEM-C_SUP_RACK1_TEMPERATURE
                H1:PEM-X_EBAY_RACK1_TEMPERATURE
                H1:PEM-X_EBAY_RACK2_TEMPERATURE
                H1:PEM-Y_EBAY_RACK1_TEMPERATURE
                H1:PEM-Y_EBAY_RACK2_TEMPERATURE
 

In the corner station electronics room the sensors are located:
CER_RACK1  below the mezzanine above space in front of ISC racks.
SUP_RACK1  above mezzanine center of the wall just below one of the air handlers.  

In the end stations

Rack1 is above the electronics racks mounted to the bottom of the cable tray
Rack2 is above the DC power supply racks mounted to the bottom of the cable tray

In the MSR
Rack1 is above Rack 2
Rack2 is above rack 11

EndX input module has a problem.  The disconnected probe did not read the same value as the others and now that the RTDs are connected the signal is glitching.  To fix this will require opening the box and replacing the module.  This will take down the entire EX beckhoff system as this is the main link to the corner

LVEA Swept

Betsy, TJ

Walked through and followed the checklist as closely as we could. There is still a power distribution box on near ISCT6 that seems to be plugged into many differenet things. Last week this was deemed OK for the time being so we left it as is. There are many cables still hanging from racks not plugged into anything. These will need to be cleaned up in a future Maintenance Day.

Lights are OFF

Monitors/Work Stations OFF

Wifi OFF

Phones disconnected and batteries taken out of handsets

Cranes in "parking spots"

Cleanrooms OFF

ISC table fans OFF

Forklifts are not charging

PRC optical gain versus lock stretch

I have been somewhat curious about how our power recycling cavity's optical gain changes throughout a lock stretch.  This is a somewhat different analysis to Sheila's from aLog 21073, where she looked at what optical gain we first lock at. 

My first attachment shows the power recycling gain for both Hanford and Livingston for the first ~21 days of O1. Hanford is green xs, Livingston is blue dots. For Hanford, I estimate the PRC gain by multiplying the Yarm transmission by the PRM transmission.  For Livingston, I estimate the PRC gain by dividing the POP DC power by the power input to the IFO.  Both of these methods assume that the signals I'm using are appropriately calibrated, but they're the same way that seems to be used at each respective site by others.  So, while I don't guarantee the absolute values of the PRC gains in my plots, the trends are what I'm looking at.  Note also that these are every lock stretch that gets as far as the power-up state, regardless of comissioning/observing intent.

I am only plotting the PRC gain at times when the IFOs have completed their respective power-up states (LHO = INCREASE_POWER, LLO = RF_LOCKED_AT_25W).    I don't have much to say about this plot other than that overall, LHO seems to be a little more consistent at getting to the same PRC gain each lock. 

Perhaps more illuminating are the following 2 plots (one per IFO), where I have lined up the maximum PRC gain from each lock stretch (t0 for each trace is the time where the maxima occurs.  The start of the traces are the time when the state following the power-up state begins).  There are 49 traces on the LHO plot, and 111 on the LLO plot, so I'm just trying to look at the general character of the traces, rather than pick any one particular trace out of each plot. 

Here at LHO we have a large spike in the recycling gain right after we power up, then we settle out.  At LLO, they seem to have a much smaller, more smooth hump before settling out, although the time constant seems to be roughly similar between both sites.  At LLO (at least according to their guardian state names) they spend some time at 10W before going to full power, so that could be the reason for the difference in character. 

new version of ext_alert.py running on h1fescript0

WP5538

the ext_alert.py code running on h1fescript0 was upgraded to version r11827

Channels used to calibrate the data added to the DCS (LDAS) rds frame files

Starting from 1128187392 GPS, the DCS (LDAS) rds frames (frame-type H1_RDS) were updated to include all the channels needed by gstlal_compute_strain to calibrate the data:

H1:CAL-DARM_CTRL_WHITEN_OUT_DQ
H1:CAL-DARM_ERR_WHITEN_OUT_DQ
H1:CAL-PCALY_RX_PD_OUT_DQ
H1:CAL-CS_LINE_SUM_DQ
H1:CAL-CS_TDEP_DARM_LINE1_REF_A_TST_REAL
H1:CAL-CS_TDEP_DARM_LINE1_REF_A_TST_IMAG
H1:CAL-CS_TDEP_DARM_LINE1_REF_A_USUM_REAL
H1:CAL-CS_TDEP_DARM_LINE1_REF_A_USUM_IMAG
H1:CAL-CS_TDEP_DARM_LINE1_REF_A_USUM_INV_REAL
H1:CAL-CS_TDEP_DARM_LINE1_REF_A_USUM_INV_IMAG
H1:CAL-CS_TDEP_REF_CLGRATIO_CTRL_REAL
H1:CAL-CS_TDEP_REF_CLGRATIO_CTRL_IMAG
H1:CAL-CS_TDEP_PCALY_LINE2_REF_C_NOCAVPOLE_REAL
H1:CAL-CS_TDEP_PCALY_LINE2_REF_C_NOCAVPOLE_IMAG
H1:CAL-CS_TDEP_PCALY_LINE2_REF_D_REAL
H1:CAL-CS_TDEP_PCALY_LINE2_REF_D_IMAG
H1:CAL-CS_TDEP_PCALY_LINE2_REF_A_TST_REAL
H1:CAL-CS_TDEP_PCALY_LINE2_REF_A_TST_IMAG
H1:CAL-CS_TDEP_PCALY_LINE2_REF_A_USUM_REAL
H1:CAL-CS_TDEP_PCALY_LINE2_REF_A_USUM_IMAG
H1:SUS-ETMY_L3_CAL_LINE_OUT_DQ
H1:CAL-CS_TDEP_REF_INVA_CLGRATIO_TST_REAL
H1:CAL-CS_TDEP_REF_INVA_CLGRATIO_TST_IMAG
H1:ODC-MASTER_CHANNEL_OUT_DQ
H1:CAL-DELTAL_CTRL_TST_DQ
H1:CAL-DELTAL_CTRL_PUM_DQ
H1:CAL-DELTAL_CTRL_UIM_DQ
H1:CAL-DELTAL_RESIDUAL_DQ
H1:CAL-DELTAL_CTRL_DQ

Some of the faster channels were already in the rds frames. This increases the rds data rate from ~2.1 MB/s to ~2.2 MB/s

The same changes was made at LLO, with H1->L1.

The H1 and L1 DCS (LDAS) rds channel list are here:

http://www.ldas.ligo-wa.caltech.edu/ldas_outgoing/createrds/channels_H1_RDS/channelList-O1-H1_RDS.txt
http://www.ldas.ligo-la.caltech.edu/ldas_outgoing/createrds/channels_L1_RDS/channelList-O1-L1_RDS.txt

Inspiral range integrand for current strain spectrum

The attached plots show the inspiral range integrand, and cumulative integral, for a stretch of recent H1 strain data. This is just the standard integration of the strain noise power, weighted as (frequency)^(-7/3). I was also interested in the impact the 35-40 Hz calibration lines have on the range calculation, so the plots include a cumulative integral curve for which the calibration lines have been artificially removed from the strain spectrum (the strain noise in the 35-38 Hz band was replaced with the average strain noise at nearby frequencies). These curves (magenta) show that the calibration lines reduce the range calculation just a bit -- by just less than 1 Mpc.

The inspiral range for the spectrum used is 75 Mpc. 90% of the total is accumulated by 150 Hz; the second plot thus shows the same data from 0-150 Hz. At the lower frequency end, 10% of the total range comes from the band 16-26 Hz.

Hi Peter,

Andy pointed me to this post, indicating that this result shows we might want to filter from lower frequencies in the PyCBC offline CBC search. However, when we run our own scripts to generate the same result we don't see nearly as much range coming from the 20-30Hz band. Instead, we see only ~1% of the inspiral range coming from this band.

Initially Andy had a script that agreed with your result, however I've convinced him that there was a bug in that script. I think that it might be possible that the same bug is also present in yours. 

I've attached a python script and a PSD from that time that should generate a relative range plot. I hope that it is clear enough to check if your scripts are doing the same thing. 

maintenance activities

This morning I did a few things:

First I made some injections to the EMTX ISI (WP5528).  This data can be used to make a projection to the normal level of noise. 

I made 3 guardian changes (WP 5533):

I added a new state to the OMC_LOCK guardian which tries to keep the OMC off resonance, and the ISC_LOCK guardian requests this for the early part of lock acquisition.

Added a function which checks that the ETM pit oplevs have an rms less than 0.5 urad before we attempt to start locking again.   This will prevent tidal from acculating junk while the optics are still swinging. 

Changed the decorators in the ISC_LOCK guardian so that we do not check the DRMI lock once CARM is locked on the transmitted power (this will reduce th confusion in lockloss logs).

I also copied a bunch of medm screens and filters in the ASC ADS system from LLO, as well as scripts with help from Carl, Adam and Marie.  This is in preparation to try using Marie's modifications to Hang's A2L script.  

ETM charge measurements

This morning, I took another set of 5 charge measurements on each ETM.  This morning was much quieter at the end stations and the measurments also ran through to completion, so the data from today look better than last weeks points.  Attached are the trends including todays new data.  The charge is hovering around ~20 V.

TCS X slewing does not produce a null for jitter coupling

I looked back at the first TCS X slewing that Kiwamu, Stefan, and I did at the start of August. It seems that during this slewing, there was no null of the IMC jitter coupling. On the other hand, we already know that the coupling passes through a null close to the beginning of each lock stretch (22208).

First, the caveats: (1) the ISS outer loop was off during this test, and (2) one can see that the shape and location of the jitter peaks is different from what we normally observe in DARM.

The first attachment is a normalized spectrogram of DARM error during the slewing. It can be seen that the jitter peaks around 250 and 350 Hz increase monotonically from start to finish.

As a check, I looked again at the complex coherence between the IMC WFS error signals and DARM at the beginning and the end of this test (at 01:19:00 and 03:19:00, 2015-08-03 Z). Unlike what we see at the start of each lock, there is no sign flip of the coupling:

This seems to indicate that thermal tuning of the X arm alone is not enough to minimize the coupling. Either the null that we see at the beginning of each lock cannot be reached by thermal tuning, or we need thermal compensation on both arms.

LHO HEPI Pressure Trends--All Pumps Operating Steady

Attached is 2 weeks of data.  No pressure fluctuations seen == Pumps running well.  Also on the plots are the Control Out from the VFD.  Looks like outside temperature trends showing at EndY and that is likely the reason for the large fluctuations on the L0 (CS) control--facility adjusting HVAC.  The daily and weekly glitches at the EndY continue unabated.

LHO HEPI Fluid Reservoirs Checked--Good

Levels steady for several weeks now.  No further maintenance required.