After Jason had to fill the TCSY chiller this morning he and I decided to make another round of water leak inspections.  We "poured" (pun intended) over the inside of the table enclosure, including removing a few table panels in order to carefully look behind some things.  We found no water leaks.  We then crawled the LVEA length of piping under the chambers and up to the wall junction and also found nothing.  We inspected the flow meter area under BSC2, nothing.  And so we keep filling a bit every dayish...

J. Betzwieser, D. Barker, D. Tuyenbayev,

Overview

Updated the h1calcs model to include modifications from LLO, restarted the DAQ system (LHO WP #6238). Now H1 and L1 are running the same version of the CAL-CS model.

Details

• CTRL_DELAY filter bank was added to the DELTAL_CTRL signal path, before the summation with DELTAL_RESIDUAL (see LLO alog 28268).
• Reverted some modifications to BUFFER_AND_AVERAGE.c, currently using version of the script in CDS SVN r14393 (see LHO alog 30293).

After the DAQ restart the coherence calculation settings were changed to:

Number of averages: H1:CAL-CS_TDEP_COH_BUFFER_SIZE 13
Seconds between averages: H1:CAL-CS_TDEP_COH_STRIDE 10

The new CTRL_DELAYfilter is currently empty (unused), and the DELAY_CYCLES (ring buffer / integer delay cycles) is set to 7.

The synchronized oscillators in the CAL-CS time-dependent parameter and coherence calculations were restarted manually. This is a necessary step to get correct phases in the oscillators after a model restart (a bug is described in LHO alog 30220).

WP 6235:

We added a feedforward path from the DBB QPDs to the LSC DARM or LSC REFL_DC_BIAS. There are new filters name H1:PSL-DBB_FF_QPD1X, 2X, 1Y and 2Y, followed by a matrix H1:PSL-DBB_FF_QPDMAT. Then the signal is sent to the LSC model, where there is a filter module H1:LSC-JITTERFF followed by an output matrix H1:LSC-JITTERFF_MAT. The signal is then added to either the DARM error point, or sent to the REFL_DC_BIAS output.

In the LSC model the temporary third loop ISS which is now defunct was deleted and the EXTRA_AO_3 path restored. The hardware was also disconnected.

We also noticed that the REFL_DC_ERR and COMM_A_RF_ERR paths were swapped. The same was true for REFL_A_RF9_ERR and REFLAIR_A_RF9_ERR. They should be correct now. The REFL_DC_BIAS output is directly connected to the REF_DC_ERR input. If we need dewhitening we should switch this to the excitation input.

WP 6239:

The model part for the second loop ISS QPD was changed to one which supports the whitening filter switching.

I restarted the primary and redundant calibration pipelines at Hanford at gps time 1160244770. Based on tests, the current latency is ~5-7 seconds. This restart picked up the latest release gstlal-calibration-1.0.5, with bug fixes for a "time-reversal" errors and (hopefully) the overlapping frame bug. The filters file used can be found in the calibration SVN:

aligocalibration/trunk/Runs/PreER10/GDSFilters/H1GDS_1159648300.npz

The new filters contain the updated calibration line frequencies, so the kappas should be correctly computed once we enter a lock stretch. For more information on the filters, see this aLOG:

https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=30210

Below is the command line used to run the pipeline:
gstlal_compute_strain
        --data-source=lvshm 
        --shared-memory-partition=$LIGOSMPART 
        --filters-file=$filter_file 
        --ifo=H1 
        --frame-duration=4 
        --frames-per-file=1 
        --write-to-shm-partition=$HOFTSMPART 
        --compression-scheme=6 
        --compression-level=3 
        --control-sample-rate=4096 
        --expected-fcc=341.0 
        --partial-calibration

To remove missing ISI & HEPI DACKILL widgets--WPs 6217 & 6227.  SUS & IOP DACKILLs not touched.

ECR E1600042 WP 6217; ECR E1500325 WP 6227

To Remove DACKILL elements from HAM ISIs and HEPIs:

HAM ISI

.) Removed DACKILL Widgets on WD Block--Attachment 1.

.) Removed DACKILL link from WD Block to the ERRMON Block--Attachment 2.

.) Routed WD_STATE from WD Block to ERRMON--Attachment 2.

.) Added WD_STATE Logic in ERRMON Block for Rogue Excitation Evaluation--Attachment 3.

HEPI

.) Removed DACKILL elements from HEPI (Top) Level--Attachment 4.

.) Removed DACKILL parts  on WD Block--like ISI Attachment 1.

To update HEPIs to Bleedoff-Accumulated-Saturations feature running on ISIs for months:

.) Add WINDOWTIME inputs on HEPI WD Block--Attachment 5.

.) Removed Test Point pickoffs WD Flag channels--also on Attachment 5.

.) Replace model saturation counter on WD/IPS, L4C & ACT Blocks with WD_SATCOUNT_vb.c counter & bleedoff Function--Attachment 6 becomes Attachment 7, e.g.

JimB restarted all these models after the build and install.  Reisolation was not a problem.  HAM2 did have one trip on CPS during 1st isolation attempt but it worked fine on the 2nd...

SVN COMMITS:

/opt/rtcds/userapps/release/isi/common/models/

hugh.radkins@opsws1:models 0$ svn commit -m "Removed DACKILL remenants & wired WD_STATE to ERRMON"
Sending        models/isihammaster.mdl
Transmitting file data .
Committed revision 14424.

/opt/rtcds/userapps/release/hpi/common/models

hugh.radkins@opsws1:models 0$ svn commit -m "Added sensor/actuator WD Saturation Bleedoff ala ISI & removed Test Point WD Flag channels"
Sending        models/hepitemplate.mdl
Transmitting file data .
Committed revision 14425.
 

Still have some medm and SDF cleanup to do.  In progress.

Corey informed me that he was getting low water level alarms on the TCS chillers.  I went out to investigate and found the following:

• TCS-X
  • No additional water required, water level 29.0
• TCS-Y
  • Found water level at 4.0; added 350 mL water to raise the water level to 9.0

WP6240:

The TwinCAT code has been updated on all interferometer Beckhoff machines. Three new features were added related to error reporting:

• The picomotor controller can now be set to nominally off. With this we can rise an error flag, when the controller is on.
• The LO level test for the demodulators can now be disabled by setting the nominal value to -100 dBm. This allows to disable the test for unconnected spare channels which tended to drift around.
• The tolerance on the EOM drive power readback has been increased to accommodate the 9MHz driver.

The svn version on the end stations is 3185, whereas the corner is 3186.

At the request of DetChar, I have reviewed the LHO PSL ODC thresholds for relevancy now that the HPO is operating.  The threshold for Bit 12 (PMC Transmission Limit OK) was still set to 11.5W, which was ~50% of the PMC transmitted power during O1 but is now too low for an operational HPO.  Keeping the 50% of output intact, I changed the threshold to 50W; I chose 50W as that is roughly 50% of our current 101W transmitted by the PMC.  I accepted the change in the PSL PMC SDF.

This completes WP #6243.

The shutter had not been working in the remote setting for the PSL DBB.  When Peter and Jason reseated the connectors on the shutter last week they fixed the problem.  They only tested the manual switch and left it open for the week. This morning I tested the remote control and the shutter open and closed as designed.

I have updated the gds software to v2.17.10 and the gstlal-calibration to v1.0.5 and restarted the dmt process manager. After a few tweaks, everything seems to be working.

11:44 NLN ~50MPc

12:55 damped PI mode 17 by by adding another -60 degrees to the phase.

12:58 Had to damp ITMX Roll mode. Changed phase to -60 and set the gain to 20.4 (incrementing as directed in the Wiki)

13:00 restarted/reset all FOMs

13:38 added another -20 degrees to the PI mode 17 phase. (-130 degrees total)

Follow up to alog 30294

I've re-measured the open loop transfer function of Mode2 a few times after decreasing gain and tweaking phase to try to move towards stability. Halving damping gain halves loop gain (31 --> 16) but neither changing damping phase (+/- 30 deg) or phase picked up from bandpass (30 deg --> 0 deg) changes overall loop phase; all still leave us with ~160 deg phase differential. 

• I had to use the CHECK_MICH_FRINGES state to get the Beam splitter re-aligned after the lock stretch that Patrick had pior to my shift. I also had to move PRM around to get flashes good enough to lock PRMI.

07:58 Back to NLN. Terra doing some excitations ~50MPc

• ITMX Roll Mode has to be damped. It got up to ~ e^-12
• There is a very annoying 'locking-up/freezing' on medm screens! I've logged in as Ops, myself and Controls. It happens under all of those accounts

10:02 Terra informs me she's about to do some pretty invasive injections

10:44 Injections caused PI to run away and break lock

10:56 put ISC-LOCK into INITIAL_ALIGNMENT to re-align green arms.

Lost more than 5 hour lock 10 minutes before end of shift. SUS OMC and HAM6 ISI tripped. Cause not immediately obvious. Handing over to Ed.

23:00 UTC Jenne is in LVEA by HAM6
23:23 UTC Gerardo back from overfilling CP3
23:27 UTC Disconnected and reconnected conlog-test-master to ~100,000 channels
23:41 UTC Starting initial alignment
23:46 UTC Changed from VERY_WINDY_NOBRSXY to WINDY. Improved X arm transmission and stability.
23:55 UTC Jenne back, whitening AS_A chassis swapped (Jenne, Mark, Filiberto)
00:21 UTC Initial alignment done. Jenne to LVEA.
00:42 UTC Jenne back. Starting relocking.
01:36 UTC NLN
01:55 UTC Jeff K. starting calibration sweeps
03:00 UTC Terra starting excitation of PI mode 2
03:18 UTC Jeff K. done
04:51 UTC Nutsinee to mezzanine to check TCS chiller
04:59 UTC Nutsinee back
05:36 UTC Jenne to LVEA to jiggle RF45 cable
06:01 UTC Jenne back
06:50 UTC Lost lock. SUS OMC and HAM6 ISI tripped.

Earlier this evening, I tried hand-tuning the a2l coefficients for PRM and SRM.  Just as I was finishing and trying to see if I actually did anything good, the RF45 went crazy, so we'll have to wait for a future lock to really start looking at coherences, etc.

• PRM Pit
  • Dither M3 at 20.125 Hz, 8 counts.
  • Change M3 P2L gain from 0 to +1.1 to minimize the peak in PRCL. 
  • No discernible change in the peak height in DARM.
  • Using P2L gain of +0.5 was better for minimizing the peak in DARM, but not for minimizing the peak in PRCL.
• PRM Yaw
  • Dither M3 at 20.125 Hz, 20 counts.
  • Change M3 Y2L gain from 0 to +0.87 to minimize peak in PRCL.
  • This makes the peak in DARM worse!  Why?
• SRM Pit
  • Dither M3 at 20.125 Hz, 10 counts.
  • Change M3 P2L gain from 0 to -1.0.
  • Minimizes peak height in SRCL.
  • Drastically reduces peak height in DARM.
• SRM Yaw
  • Dither M3 at 20.125 Hz, 10 counts.
  • Change M3 P2L gain from 0 to -1.2.
  • Minimizes peak height in SRCL.
  • Drastically reduces peak height in DARM.

After this, I ran A2L for the test masses.  Then RF45 went bad.  So, I'll come back to this tomorrow.

About an hour ago, I noticed that our sensitivity degraded significantly in the bucket.  This brought our range down to about 20-30 Mpc from the 50+ Mpc we had been at.

Glancing up, I noticed that the RF45 spectrum looks atrocious.  I'm not at all sure what is going on, but it is very bad, and changing with time.

In the attached screenshot, I have the DARM spectrum, the RF 45 and RF 9 spectra, as well as the RF coherences with DARM.  The colors are the same in each plot, with the start times of the data indicated in the DARM legend.  Dark blue is a good time, when our range was good.  Pink and green and red show different times when there was a weird peak in the RF45 spectra.  Brown and light blue are intermediate times when the RF45 is elevated, but not peaky. 

I don't know why this would have anything to do with the new barrel connector that Richard et al. put in earlier today (alog 30375), but maybe I'll go jiggle it.

EDIT:  Before any jiggling, Terra Patrick and I notice that it's steadily getting worse :(

03:35 UTC Connecting to ~100,000 channels from conlog-test-master.