J. Kissel, B. Weaver
WP #5444
ECR E1500346
II 1097

We've completed the removal of the the violin mode monitoring from the QUAD models this morning. See attached model change screen shots. 

The good news: It has reduced the end-station's QUAD model's CPU clock-cycle turn-around-time from a range of (57 - 67) to a range of (53 - 61); see attached trend (DCUID 88 = ETMX, DCUID 98 = ETMY), which puts these models back under the 1/16384 [hz] = 61 [us] limit, so we should see a drastic reduction in clock-cycle overage timing errors.

The bad news, in the copy over, we've "lost" all of the band-limiting and low pass filters. They're not truely lost, because they're in the filter archive, but because the filter banks no longer exist in the QUAD models, they don't exist in the current foton file, so it's going to be an arduous, by-hand copy and paste between files. Nutsinee and Betsy will work on restoring these tomorrow.

The new front-end library part that is now a part for the violin mode damping that now lives in the OAF model:
/opt/rtcds/userapps/release/sus/common/models/DARMERR_VIOLIN_MODE_MONITOR_MASTER.mdl
which has been committed to the repo. I've also committed the new local version of the top-level model,
/opt/rtcds/userapps/release/isc/h1/models/h1oaf.mdl

They've been removed from this sub-library part of the QUAD_MASTER library:
/opt/rtcds/userapps/release/sus/common/models/FOUROSEM_DAMPED_STAGE_MASTER_WITH_DAMP_MODE.mdl

This closes out the above mentioned ECR and Integration Issue for LHO. 
For LLO to update:
(1) Update the above mention library SUS common library folder
(2) Update the LLO copy of the h1oaf.mdl model for demonstrative purposes
(3) Install SUS block as shown into l1oaf.mdl
(4) The new(ish) screens (see LHO aLOG 20378
for these can be found here:
/opt/rtcds/userapps/release/sus/common/medm/quad/
SUS_CUST_QUAD_L2_VIOLIN_MON_ALL.adl
SUS_CUST_QUAD_L2_VIOLIN_MON.adl

I've also saved new SDF tables such that the channels moved from the QUAD are no longer NOT FOUND.

We started off the morning with immediate PSL work (20638) and SR3 T2 OSEM troubleshooting (20618 requiring a full coil driver swap).  As part of the diagnostic of SR3 T2 OSEM issue, we recalibrated the DAQs on HAM5/6 (which involved restarting all of the user models on h1sush56.)

From there, we performed the planned restarts of the 4 QUAD models, the ODC model, and the OAF model.  While restoring the QUADs from SAFE to ALIGNED after the reboots, we found a bug in the SUS Guardians that caused them to go into error and not complete the transition (20627).  Kissel ended up putting the Guardians into Manual and toggling through states to finish the job.

However, the subsequent DAQ restart did not go well (20637), and we had to pause for a bit Dave to fix.

While PSL PMC work continued, various other planned tasks occured:

- Phone work at the EX

- TCSY Temp Sensor install

- HWS power supply swap

- EY/EX PEM Patch Panel and magnetometer work

- Jim HEPI filter investigations

- Kiwamu ALS DIFF PLL characterization

- ETM charge measurements

Around noon we started initial alignment.  We immediately ran into problems with:

- All 4 QUAD PUM coil watchdogs tripped due to some of the DARM DAMP VIOLIN MODE loop gains restoring to non-zero values and blasting the PUM with junk - I zeroed out these gains in the SAFE.snap files so this doesn't happen again after restarts.  The ISC_LOCK Guardian should set the gains appropriately.

- ITMx PUM had 1 coil watchdog trip which we could not clear from the medm - the PUM chassis was power cycled

- Found that a critical filter in the ITMy M0 LOCK P and Y backs were not restored to being ON after the front end restarts today - if the -140dB FM5 filter is not engaged in these loops the SUS gets a crazy kick due to giant error signals coming in from ASC during WFS steps of locking.  I have captured tha these filters get switched on in the SAFE.snap for future reboots.

- Found ETMy ESD latched to -16k  (20644)

- Found ETMx ESD not in a correct state - toggling HV ON/OFF caused it to latch to -16k (20652)

- Found ALS polarization is incorrect - bit of a stall while we stared at small error messages on the ALS screen - team commiss retuned the polar in the MSR.

So, in summary - we had 9 unexpected things happen today which took alot of troubleshooting time to diagnose and fix.

Back to continued locking - Jenne, Sheila, TJ are working on it. 

To be continued...

and finally to complete the set, here are the svn status of the safe.snap files.
All safe.snap files in the target/NAME/NAMEepics/burt/ directories are symbolic links to the userapps area.

M       /opt/rtcds/userapps/release/als/h1/burtfiles/h1alsex_safe.snap
M       /opt/rtcds/userapps/release/als/h1/burtfiles/h1alsey_safe.snap
M       /opt/rtcds/userapps/release/asc/h1/burtfiles/h1ascimc_safe.snap
M       /opt/rtcds/userapps/release/asc/h1/burtfiles/h1asc_safe.snap
M       /opt/rtcds/userapps/release/cal/h1/burtfiles/h1calcs_safe.snap
M       /opt/rtcds/userapps/release/cal/h1/burtfiles/h1calex_safe.snap
M       /opt/rtcds/userapps/release/cal/h1/burtfiles/h1caley_safe.snap
M       /opt/rtcds/userapps/release/cds/h1/burtfiles/h1ioppemmx/safe.snap
M       /opt/rtcds/userapps/release/hpi/h1/burtfiles/h1hpiham1_safe.snap
M       /opt/rtcds/userapps/release/isc/h1/burtfiles/h1iscex_safe.snap
M       /opt/rtcds/userapps/release/isc/h1/burtfiles/h1iscey_safe.snap
M       /opt/rtcds/userapps/release/isc/h1/burtfiles/h1oaf_safe.snap
M       /opt/rtcds/userapps/release/lsc/h1/burtfiles/h1lscaux_safe.snap
M       /opt/rtcds/userapps/release/lsc/h1/burtfiles/h1lsc_safe.snap
M       /opt/rtcds/userapps/release/omc/h1/burtfiles/h1omc_safe.snap
M       /opt/rtcds/userapps/release/pem/h1/burtfiles/h1pemcs_safe.snap
M       /opt/rtcds/userapps/release/pem/h1/burtfiles/h1pemmx_safe.snap
M       /opt/rtcds/userapps/release/pem/h1/burtfiles/h1pemmy_safe.snap
M       /opt/rtcds/userapps/release/psl/h1/burtfiles/dbb/h1psldbb_safe.snap
M       /opt/rtcds/userapps/release/psl/h1/burtfiles/fss/h1pslfss_safe.snap
M       /opt/rtcds/userapps/release/psl/h1/burtfiles/iss/h1psliss_safe.snap
M       /opt/rtcds/userapps/release/psl/h1/burtfiles/pmc/h1pslpmc_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1susbs_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1susetmx_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1susetmy_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1sushtts_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1susim_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1susitmx_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1susitmy_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1susmc1_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1susmc2_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1susmc3_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1susomc_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1suspr2_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1suspr3_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1susprm_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1sussr2_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1sussr3_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1sussrm_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1sustmsx_safe.snap
M       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1sustmsy_safe.snap
?       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1susetmxpi_safe.snap
?       /opt/rtcds/userapps/release/sus/h1/burtfiles/h1susetmypi_safe.snap

Here is the current SVN status of the front end filter file's svn status.

A reminder, in the /opt/rtcds/lho/h1/chans/ directory, all the filter files should be symbolic links to the appropriate userapps filterfiles area.

I found that the following chans directory files are not symbolic links:

H1CAL[CS,EX,EY].txt, H1SUSETM[X,Y]PI.txt

I found that all symbolic links used the absolute path of /opt/rtcds/userapps/release/ except for:

../../../userapps/release/sus/h1/filterfiles/H1SUSBS.txt

Here is the list of the svn status:

M       /opt/rtcds/userapps/release/hpi/h1/filterfiles/H1HPIHAM1.txt
M       /opt/rtcds/userapps/release/hpi/h1/filterfiles/H1HPIHAM6.txt
M       /opt/rtcds/userapps/release/hpi/h1/filterfiles/H1HPIHAM2.txt
M       /opt/rtcds/userapps/release/hpi/h1/filterfiles/H1HPIHAM3.txt
M       /opt/rtcds/userapps/release/isi/h1/filterfiles/H1ISIHAM3.txt
M       /opt/rtcds/userapps/release/hpi/h1/filterfiles/H1HPIHAM4.txt
M       /opt/rtcds/userapps/release/hpi/h1/filterfiles/H1HPIHAM5.txt
M       /opt/rtcds/userapps/release/hpi/h1/filterfiles/H1HPIITMY.txt
M       /opt/rtcds/userapps/release/hpi/h1/filterfiles/H1HPIBS.txt
M       /opt/rtcds/userapps/release/hpi/h1/filterfiles/H1HPIITMX.txt
M       /opt/rtcds/userapps/release/sus/h1/filterfiles/H1SUSPRM.txt
M       /opt/rtcds/userapps/release/sus/h1/filterfiles/H1SUSPR3.txt
M       /opt/rtcds/userapps/release/sus/h1/filterfiles/H1SUSSRM.txt
M       /opt/rtcds/userapps/release/sus/h1/filterfiles/H1SUSSR3.txt
M       /opt/rtcds/userapps/release/sus/h1/filterfiles/H1SUSOMC.txt
M       /opt/rtcds/userapps/release/sus/h1/filterfiles/H1SUSITMY.txt
M       ../../../userapps/release/sus/h1/filterfiles/H1SUSBS.txt
M       /opt/rtcds/userapps/release/sus/h1/filterfiles/H1SUSITMX.txt
M       /opt/rtcds/userapps/release/pem/h1/filterfiles/H1PEMCS.txt
M       /opt/rtcds/userapps/release/isc/h1/filterfiles/H1OAF.txt
M       /opt/rtcds/userapps/release/lsc/h1/filterfiles/H1LSC.txt
M       /opt/rtcds/userapps/release/omc/h1/filterfiles/H1OMC.txt
M       /opt/rtcds/userapps/release/lsc/h1/filterfiles/H1LSCAUX.txt
M       /opt/rtcds/userapps/release/asc/h1/filterfiles/H1ASC.txt
M       /opt/rtcds/userapps/release/asc/h1/filterfiles/H1ASCIMC.txt
M       /opt/rtcds/userapps/release/pem/h1/filterfiles/H1PEMMX.txt
M       /opt/rtcds/userapps/release/pem/h1/filterfiles/H1PEMMY.txt
M       /opt/rtcds/userapps/release/sus/h1/filterfiles/H1SUSETMY.txt
M       /opt/rtcds/userapps/release/hpi/h1/filterfiles/H1HPIETMY.txt
M       /opt/rtcds/userapps/release/isc/h1/filterfiles/H1ISCEY.txt
M       /opt/rtcds/userapps/release/sus/h1/filterfiles/H1SUSETMX.txt
M       /opt/rtcds/userapps/release/hpi/h1/filterfiles/H1HPIETMX.txt

and here are the scripts which generated it (running in the chans directory):

for i in 'awk '{print $1}' ../cds/model_info.txt';do echo $i|tr [a-z] [A-Z]|awk '{print $i".txt"}'>>/tmp/filtfilelist.txt;done

for i in 'cat /tmp/filtfilelist.txt' ;do ls -al $i >> /tmp/filelonglisting.txt;done

for i in 'grep  "->" /tmp/filelonglisting.txt |awk 'BEGIN{FS=" -> "}{print $2}'';do svn st $i;done

I have checked the SVN status of the source files used to build the front end models.

To do this, from the /opt/rtcds/lho/h1/target directory I ran the script

for i in 'sort -u */src/src_locations.txt';do svn st $i;done

This produced the output:

M       /opt/rtcds/userapps/release/cal/h1/models/h1calex.mdl
M       /opt/rtcds/userapps/release/cal/h1/models/h1caley.mdl
M       /opt/rtcds/userapps/release/isc/h1/models/h1iscex.mdl
M       /opt/rtcds/userapps/release/isc/h1/models/h1oaf.mdl
?       /opt/rtcds/userapps/release/sus/common/models/DARMERR_VIOLIN_MODE_MONITOR_MASTER.mdl
M       /opt/rtcds/userapps/release/sus/common/models/FOUROSEM_DAMPED_STAGE_MASTER_WITH_DAMP_MODE.mdl

On first run through I saw that my h1iopsuse[x,y] changes to add the additional ADC card for PI were not committed, so I have taken care of that.

Yesterday I got a chance to look at the phasing of REFL 45 WFS again, and to measure the sensing matrix for the REFL wfs with a new phasing.  In the end, these phasings were reverted so we could go back to running without recommisioning the loops.

I drove a line in CHARD pit, and phased the 45 WFS quadrant by quadrant to minimize the CHARD signal in I.

The phases of the CHARD pit signals have the appropriate phases for PIT with both the current settings and the ones that minimize the signal in I.  The motivation for doing this was to get a sensing matrix that would allow us to distinguish between CHARD and PR3 motion.  After rephasing I drove a line at 8 Hz in CHARD, IM4, and PR3 and measured the transfer function from the excitation to REFL WFS. For reference, I drove 3 Counts in CHARD P EXC, 100 counts inPR3 M3 drivealing P2P, and 1 count in IM4 M1 drivealign P2P.  The amplitudes of the lines were choosing to give good coherence, and I have not calibrated this measurement in any units, so only comparisons down the colmuns in this table are meaningful:

With these phasings, we should be able to use something in our input matrix like:

A9I+B9I-2*(A45I+B45I) to CHARD which would reduce the coupling of PR3 motion into CHARD which could be causing us some grief (alog 20523.) Currently we use only A9I+B9I.

A45I+B45I for PR3 should be a better option than it used to be for the PR3 loop, although we may still need some 9I terms to cancel CHARD. 

We can try closing loops around these new phases and adjusting our input matrix next time there is a chance. 

SudarshanK, FilbertoC

We installed an accelerometer at ISCTEY  (ISC table at ENDY) and a patch panel (D1300632) that routes the cable from outside the table to the inside. The corresponding channel for this accelerometer is H1:PEM-EY_ACC_ISCTEY_TRANS_X. The first plot is the spectrum of the newly installed accelerometer.

At ENDY, we installed a patch panel and routed the accelerometer (H1:PEM-EX_ACC_ISCTEX_TRANS_Y) through the patch panel on ISCTEX . The second plot is the spectrum before and after the installation.

Both patch panel were installed on the right side of the table on slot 3-7 which had a quintiple blank (D1100422) to start with. The optical table feedthrough layout document D1101126 for EY and D1201495 for EX needs to updated accordingly. Pictures attached.

No replacement of controller for IP-03 took place, the old Multivac controller decided to start working as I was retrieving settings, old controller stays, for now.

Maintenance day, hooray!!

14:59 Richard to CER

15:14 Peter, Jason to H1 PSL enclosure for PMC work, then to H2 PSL to shut down for O1

15:16 Carlos to EX for phone work

15:21 JimW taking HAM1 HEPI down for blend filter work

15:21 Daniel to MY doing what Daniel does

15:48 Kiwamu to LVEA for VCO measurements

15:48 Duncan restarting ODC model

16:02 Fil and Elli to LVEA TCS table near HAM4

16:05 Jodi, Mitchell, Kyle to all VEAs for property tagging

16:09 Mike and BBC crew to LVEA

16:14 DAQ restart

16:16 Daniel done

16:19 Sudarshan to EY for PEM patch panel/accelerometer work

16:47 Elli and Fil done

16:49 Karen and Cristina to LVEA

17:26 Mike and BBC crew to EX

17:33 Sudarshan done

17:36 PSL crew done

17:49 Fil done

17:51 Sudarshan and FIl to EX

18:17 Jodi, etc. to LVEA

19:22 Jodi, etc. done

20:00 Sudarshan and Fil done

20:13 Fil back

20:35 Initial alignment ongoing

20:35 Gerardo to LVEA part hunting

20:35 RIchard to EY to fix railed ESD

20:40 Gerardo out

22:00 Richard to EX fixing ESD

My bad, after the round of SSH fixes to get the 2FA auth running again on cdsssh and cdslogin I forgot to restart the cell phone texter service on cdslogin. Gerardo noticed that we didn't get our daily 1pm vacuum summary texts today, which lead to the discovery that the service had been down since 4pm yesterday.

During troubleshooting of various stages of relock this afternoon, we noticed the ETMx ESD looked "funny" so Richard headed out to EX this time in search of a cure.  By "funny" we mean the 4 QUAD signals were the same sign as the DC - normally they are opposite.  As well, when Richard got to the end station, he found the HV ON/OFF switch was being toggled (audible at the rack) repeatedly so he unplugged the Binary cable which stopped it.  He toggled the HV ON/OFF a few times to see about clearing the sign issue.  Then, after a round of conversation, we found the BIO L3 HI/LO VOLTAGE and HI VOLT DISCONNECT were in the OFF positions on the ETMx - this is an incorrect state for acquisition.  So, I have written into the SAFE.snap to restore the ETMx to the HI Voltage state after a restart.  AFter all of the rack manipulations and a correct set of MEDM switches were thrown, the ETMx ESD seems healthy.  On with locking attempts.

Chris S.

The aluminum material for the beam tube enclosure repair arrived and Chris has started cutting it to length and punching holes for install.

Scott L. Ed P.

8/17/15
48.2 meters of tube and bellows cleaned ending 8.4 meters east of HSW-2-002.

8/18/15
20.8 meters of tube cleaned ending at the west wall of Mid-Y. 
The crew will stop at this point until after the O1 Run. All equipment and cords will be thoroughly cleaned and put away for storage.   

I have added the CAL (Maddie) and DETCHAR (Ryan F) requested channels to the H1BROADCAST0.ini file and restarted h1broadcast0 to use the new configuration. 140 channels were added, none removed. The list of added channels are attached.

H1BROADCAST0.ini was committed to svn as version r11327

One of the things that is slow and not optimized well about our inital alingment is the lock of the laser to the Xarm.  This is not a hard problem, just one that we have never taken the time to work on.  This was one of the problems (annoyances) that the relocking team ran into this afternoon.  I had a look at the data from a sucsessfull acquisition, and can see two problems that have simple solutions. 

1) It takes too long to ramp down the feedback from the IMC locking loop.  We have a 3 second pause and a 2 second ramp on this gain.  While both loops are on, they fight each other and the lock is unstable (see attached screenshot).  I would suggest something like a half second for each of these, or even less. 

2)We had a 0.1 second delay on the FM triggering that turns on boosts.  The other day I had reduced this from 0.2, but it is still too long.  We should probably try a shorter FM delay or no delay.

J. Kissel, D. Barker

The state of the H1PSLISS.txt foton file was in a state of confusion. A diff between the last several back ups, shows very small changes (the 14th significant digit) to the gain coefficient in about 12 filter modules. Sudarashan ensures use that he's only been making changed to the 
ISS_SECONDLOOP_SUM14_DC
ISS_SECONDLOOP_SUM58_DC
filter banks, yet there are these small changes to several filter modules in the following banks reported over the past few backups:
ISS_SECONDLOOP_QPD_SUM
ISS_PDB_LSD_INTEGRATION
ISS_PDB_LSD_BANDPASS
ISS_PDB_CALI_AC
ISS_OSCILLATION_MON_LP

We don't understand it, and we can't trace it. However, because the changes in the uncertain filter banks appear to all be small, we've decided to just "start from scratch" and load the whole filter en masse.

So, the current archive, in 
/opt/rtcds/lho/h1/chans/filter_archive/h1psliss/
H1PSLISS_1123967724.txt
represents the currently loaded filter coefficients. Further, the current filter file, in
/opt/rtcds/lho/h1/chans/H1PSLISS.txt

which is a soft link to the userapps, 
/opt/rtcds/userapps/release/psl/h1/filterfiles/H1PSLISS.txt
has been committed to the userapps repo.

Kiwamu, Hang

We investigated the nonlinearity in the esd control signal using channel 'SUS-ETMY_L3_MASTER_OUT_UL_DQ'. In the upper panel of the attached plot, the 'linear' is the product of control signal and the bias, 'quadratic' the square of control signal, and 'total' the sum of these two. We did not do an absolute calibration but the relative magnitude should be good enough for the purpose of this study. In the lower panel we showed the coherence between the linear and quaduartic timestreams, with 512s of data and 64 averages.

From the result we can see that the nonlinear term is well below the linear component, and the coherence was low. As a result, it should be safe to use the current ESD configuration without worrying too much about the quadratic term.

Mid maintenance, the ETMy ESD tripped off.  I toggled the HV ON/OFF switch on the SUS screen which tunred it on, but in the railed ~-16k state.  I toggled the switch slowly a few times but the railing never cleared.  Richard went to EY and did the old "Power OFF, pull DAQ plug, re-power ON, re-plug in DAQ cable" trick.  It's back now.

just when we thought we had fixed the monit autostart of the daqd process on h1dc0 it again failed to start up after a DAQ restart this morning (following model changes). Using the monit web page, I was able to stop and start the daqd process manually. I verified there were no duplicate processes running and the PID file was correct. There is a DAQ frame gap due to this problem from 09:13 to 09:51 PDT (between GPS times 1123949568 - 1123951808).

Evan, Dan, Jeff

The SR3 M1 T2 actuator is...unwell.

We started to have trouble staying locked after power up about four hours ago.  At first it looked like an ASC problem in the SRC, and since the AS36 loops are the scapegoat du jour, Evan started to retune the SRC1 loop.  He observed a recurring transient in the beam spots at the AS port, and saw the same thing in the SR3 oplevs.  We turned off the SR3 cage servo and the kicks kept on coming.

Eventually we looked at the SR3 M1 voltmons and found the first plot attached, which is for eight hours.  The T2 OSEM and voltmon started to get ratty about three and a half hours ago.  The noise is a series of slow transients with a several-second rise and a steep decay.  See Fig. 2.

We're pretty convinced that it's an actuator problem, something between the DAC and the voltmon readback in the coil driver box.  We have unlocked the IFO and turned off the SR3 top-stage damping and we still see the pattern of noise.  We shall leave the pleasure of power-cycling the AI chassis and coil driver to others.

As an amuse bouche for the maintenance day team, we also discovered that the SR3 M1 T1 voltmon is complete nonsense.  The T1 voltmon time series is a collection of step functions and spikes, 100x larger than the other M1 voltmons.

The SR3 M1 damping has been turned back on.