State of H1:  in Maintenance, 18 Work Permits in progress / done

Activities: (Corey 8:00-10:38(15:00-17:38UTC))

• 8:00 LVEA Cleaning (Christina, Karen)
• 8:16 Transitioned EX & EY ISIs to "VERY_WINDY_NOBRSXY" in prep for Maintenance
• 8:22 EY to LASER HAZARD
• 8:22 PCal EY work (Travis/Pcal Crew)
• 8:25 Turbo pump controllers energizing (charge batteries), maintenance (Kyle)
• 8:43 -11? EY Vent/Purch air skid work (WP6241)  (Kyle & gerardo)
• 8:33-8:50 COMPUTERS FROZE!  (Batch & Richard looking?)
• 8:45 Cleanroom curtains, HAM3/4 (Jeff)
• 8:48 Contractor for mats
• 9:09 SEI model work (Hugh, 6217, 6227)
• 9:00 PSL (Daniel, WP6239)
• 9:14 PSL Room work (Peter, Kiwamu)
• 9:40 LN Truck at CP5 & CP3
• 9:40 Gate Valve Measurements (Chandra)
• 9:52 King Soft for RO work (Bubba)
• 9:45 Check PSL DBB shutter (Richard)
• 10:11 Paradise Water 
• 10:16 quick PSL ODC work (Jason)
• 10:08:  TCSx CO2 laser & Xand Y Chiller alarms---not sure why???  (Daniel & Jason checking)----> Jason brought back 
• 10:38 DAQ Restart

As of 18:00UTC (11:00PT) Work in progress: (Cheryl)

•  1  6234         LVEA/EX    filiberto.EE                    hardware WD elec                 x202   00:00:36
•  2  6228         EY     travis.pcal                           PCal work                 x239   00:02:09
•  3  6241         EY            kyle                 vent purge air work                 x239   00:00:14
•  4   n/a            PSL    peter.kiwamu                       PSL room work                 x113   00:00:53
•  5  6232         CDS        jonathan                    framewriter work                 x202   00:02:36
•  6   n/a            MY    liquid.nitro                            CP5 fill                 x202   00:00:38
•  7   n/a            LVEA  jason.oberling       check TCSx laser and chillers                 x202   00:00:56
•  8  6236         LVEA     dave.barker                         SUS WD work                 x202   00:01:16
•  9  6238         LVEA    dave.darkhan                        CALCS update                 x202   00:01:17
• 10  6234       EY/EX    filiberto.EE                    hardware WD elec                 x202   00:02:19
• 11   n/a          MY    liquid.nitro                            CP3 fill                 x202   00:01:23
• 12  6239        PSL-ISS           DB+DS            psliss code, DAQ restart                  255   00:03:29
• 13  6235        PSLDBB-LSC           DB+DS    dbb and lsc restart, daq restart                  255   00:03:30
• 14  6240        DAQ           DB+DS    restart DAQ for Beckhoff changes                  255   00:03:31
• 15  6231        DMT-GDS           JohnZ                     upgrade GDS DMT                  255   00:03:34
• 16  0000        lvea           bubba                                3ifo                  202   00:01:44
• 17  0000        lvea     betsy-jason                     tcsy-water-leak                  202   00:01:57
• 18  0000        lvea    richard-mike                  check-voltage-popA                  202   00:01:57

Activities Since (UTC):

• 18:37 - 6229 cds dave-kiwamu tcs-code-upgrade - done
• 18:40 - 6239 cds dave-daniel ISS-code-upgrade - done
• 18:42 - 6244 cds richard-mike popx-whitening-test-swap - no swap, connection/cable
• 19:13 - 6231 DMT-GDS JohnZ upgrade GDS DMT - done
• 19:13 - VAC - Gerardo - pimps in LVEA turned off
• 19:18 - 6241  EY kyle vent purge air work - done
• 19:21 - 6242 cds carlos migrate-software - done
• 19:21 - 6244 cds richard-mike popx-whitening-test-swap - hardware is done for now, handing back to Daniel
• 19:44 - REFL - Daniel - check signals
• 19:45 - 6234 EY/EX filiberto.EE hardware WD elec - done
• 19:47 - 6236 LVEA dave.barker SUS WD work - done
• 19:49 - LVEA  jason.oberling check TCSx laser and chillers - done
• 19:49 - lvea  betsy-jason tcsy-water-leak - done
• 20:00 - DAQ restart
• 20:00 - 6244 cds richard-mike popx-hardware - back out
• 20:07 - DAQ restart
• 20:14 - 6239 psl kiwamu measure-iss-loop - done
• 20:14 - lvea daniel refl - back
• 20:24 - lvea daniel refl

Work that continues:

•  REFL - Daniel - LVEA
•  Pcal - Rick/Travis - EY

I selected nine periods of ten minutes during last night's locks (see fig. 1). For each period I computed the transfer function from jitter (as measured by the diagnostic brteadboard Q1Y signal) to DARM. The second figure shows that the coupling indeed changes over time. 

I used the period martked as (4), which correspond to the largest coupling, fit a transfer function and used it in the time domain to perform jitter subtraction for all nine periods. The third figure shows the reduction in DARM noise: clearly the filter works well for period (4), but the performance is significantly degraded for the other periods.

In conclusion, if we want to subtract the jitter noise, we'll need some level of adaptive filtering.

After Jason had to fill the TCSY chiller this morning he and I decided to make another round of water leak inspections.  We "poured" (pun intended) over the inside of the table enclosure, including removing a few table panels in order to carefully look behind some things.  We found no water leaks.  We then crawled the LVEA length of piping under the chambers and up to the wall junction and also found nothing.  We inspected the flow meter area under BSC2, nothing.  And so we keep filling a bit every dayish...

J. Betzwieser, D. Barker, D. Tuyenbayev,

Overview

Updated the h1calcs model to include modifications from LLO, restarted the DAQ system (LHO WP #6238). Now H1 and L1 are running the same version of the CAL-CS model.

Details

• CTRL_DELAY filter bank was added to the DELTAL_CTRL signal path, before the summation with DELTAL_RESIDUAL (see LLO alog 28268).
• Reverted some modifications to BUFFER_AND_AVERAGE.c, currently using version of the script in CDS SVN r14393 (see LHO alog 30293).

After the DAQ restart the coherence calculation settings were changed to:

Number of averages: H1:CAL-CS_TDEP_COH_BUFFER_SIZE 13
Seconds between averages: H1:CAL-CS_TDEP_COH_STRIDE 10

The new CTRL_DELAYfilter is currently empty (unused), and the DELAY_CYCLES (ring buffer / integer delay cycles) is set to 7.

The synchronized oscillators in the CAL-CS time-dependent parameter and coherence calculations were restarted manually. This is a necessary step to get correct phases in the oscillators after a model restart (a bug is described in LHO alog 30220).

WP 6235:

We added a feedforward path from the DBB QPDs to the LSC DARM or LSC REFL_DC_BIAS. There are new filters name H1:PSL-DBB_FF_QPD1X, 2X, 1Y and 2Y, followed by a matrix H1:PSL-DBB_FF_QPDMAT. Then the signal is sent to the LSC model, where there is a filter module H1:LSC-JITTERFF followed by an output matrix H1:LSC-JITTERFF_MAT. The signal is then added to either the DARM error point, or sent to the REFL_DC_BIAS output.

In the LSC model the temporary third loop ISS which is now defunct was deleted and the EXTRA_AO_3 path restored. The hardware was also disconnected.

We also noticed that the REFL_DC_ERR and COMM_A_RF_ERR paths were swapped. The same was true for REFL_A_RF9_ERR and REFLAIR_A_RF9_ERR. They should be correct now. The REFL_DC_BIAS output is directly connected to the REF_DC_ERR input. If we need dewhitening we should switch this to the excitation input.

WP 6239:

The model part for the second loop ISS QPD was changed to one which supports the whitening filter switching.

I restarted the primary and redundant calibration pipelines at Hanford at gps time 1160244770. Based on tests, the current latency is ~5-7 seconds. This restart picked up the latest release gstlal-calibration-1.0.5, with bug fixes for a "time-reversal" errors and (hopefully) the overlapping frame bug. The filters file used can be found in the calibration SVN:

aligocalibration/trunk/Runs/PreER10/GDSFilters/H1GDS_1159648300.npz

The new filters contain the updated calibration line frequencies, so the kappas should be correctly computed once we enter a lock stretch. For more information on the filters, see this aLOG:

https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=30210

Below is the command line used to run the pipeline:
gstlal_compute_strain
        --data-source=lvshm 
        --shared-memory-partition=$LIGOSMPART 
        --filters-file=$filter_file 
        --ifo=H1 
        --frame-duration=4 
        --frames-per-file=1 
        --write-to-shm-partition=$HOFTSMPART 
        --compression-scheme=6 
        --compression-level=3 
        --control-sample-rate=4096 
        --expected-fcc=341.0 
        --partial-calibration

To remove missing ISI & HEPI DACKILL widgets--WPs 6217 & 6227.  SUS & IOP DACKILLs not touched.

ECR E1600042 WP 6217; ECR E1500325 WP 6227

To Remove DACKILL elements from HAM ISIs and HEPIs:

HAM ISI

.) Removed DACKILL Widgets on WD Block--Attachment 1.

.) Removed DACKILL link from WD Block to the ERRMON Block--Attachment 2.

.) Routed WD_STATE from WD Block to ERRMON--Attachment 2.

.) Added WD_STATE Logic in ERRMON Block for Rogue Excitation Evaluation--Attachment 3.

HEPI

.) Removed DACKILL elements from HEPI (Top) Level--Attachment 4.

.) Removed DACKILL parts  on WD Block--like ISI Attachment 1.

To update HEPIs to Bleedoff-Accumulated-Saturations feature running on ISIs for months:

.) Add WINDOWTIME inputs on HEPI WD Block--Attachment 5.

.) Removed Test Point pickoffs WD Flag channels--also on Attachment 5.

.) Replace model saturation counter on WD/IPS, L4C & ACT Blocks with WD_SATCOUNT_vb.c counter & bleedoff Function--Attachment 6 becomes Attachment 7, e.g.

JimB restarted all these models after the build and install.  Reisolation was not a problem.  HAM2 did have one trip on CPS during 1st isolation attempt but it worked fine on the 2nd...

SVN COMMITS:

/opt/rtcds/userapps/release/isi/common/models/

hugh.radkins@opsws1:models 0$ svn commit -m "Removed DACKILL remenants & wired WD_STATE to ERRMON"
Sending        models/isihammaster.mdl
Transmitting file data .
Committed revision 14424.

/opt/rtcds/userapps/release/hpi/common/models

hugh.radkins@opsws1:models 0$ svn commit -m "Added sensor/actuator WD Saturation Bleedoff ala ISI & removed Test Point WD Flag channels"
Sending        models/hepitemplate.mdl
Transmitting file data .
Committed revision 14425.
 

Still have some medm and SDF cleanup to do.  In progress.

Corey informed me that he was getting low water level alarms on the TCS chillers.  I went out to investigate and found the following:

• TCS-X
  • No additional water required, water level 29.0
• TCS-Y
  • Found water level at 4.0; added 350 mL water to raise the water level to 9.0

WP6240:

The TwinCAT code has been updated on all interferometer Beckhoff machines. Three new features were added related to error reporting:

• The picomotor controller can now be set to nominally off. With this we can rise an error flag, when the controller is on.
• The LO level test for the demodulators can now be disabled by setting the nominal value to -100 dBm. This allows to disable the test for unconnected spare channels which tended to drift around.
• The tolerance on the EOM drive power readback has been increased to accommodate the 9MHz driver.

The svn version on the end stations is 3185, whereas the corner is 3186.

At the request of DetChar, I have reviewed the LHO PSL ODC thresholds for relevancy now that the HPO is operating.  The threshold for Bit 12 (PMC Transmission Limit OK) was still set to 11.5W, which was ~50% of the PMC transmitted power during O1 but is now too low for an operational HPO.  Keeping the 50% of output intact, I changed the threshold to 50W; I chose 50W as that is roughly 50% of our current 101W transmitted by the PMC.  I accepted the change in the PSL PMC SDF.

This completes WP #6243.

The shutter had not been working in the remote setting for the PSL DBB.  When Peter and Jason reseated the connectors on the shutter last week they fixed the problem.  They only tested the manual switch and left it open for the week. This morning I tested the remote control and the shutter open and closed as designed.

I have updated the gds software to v2.17.10 and the gstlal-calibration to v1.0.5 and restarted the dmt process manager. After a few tweaks, everything seems to be working.

11:44 NLN ~50MPc

12:55 damped PI mode 17 by by adding another -60 degrees to the phase.

12:58 Had to damp ITMX Roll mode. Changed phase to -60 and set the gain to 20.4 (incrementing as directed in the Wiki)

13:00 restarted/reset all FOMs

13:38 added another -20 degrees to the PI mode 17 phase. (-130 degrees total)

Follow up to alog 30294

I've re-measured the open loop transfer function of Mode2 a few times after decreasing gain and tweaking phase to try to move towards stability. Halving damping gain halves loop gain (31 --> 16) but neither changing damping phase (+/- 30 deg) or phase picked up from bandpass (30 deg --> 0 deg) changes overall loop phase; all still leave us with ~160 deg phase differential. 

• I had to use the CHECK_MICH_FRINGES state to get the Beam splitter re-aligned after the lock stretch that Patrick had pior to my shift. I also had to move PRM around to get flashes good enough to lock PRMI.

07:58 Back to NLN. Terra doing some excitations ~50MPc

• ITMX Roll Mode has to be damped. It got up to ~ e^-12
• There is a very annoying 'locking-up/freezing' on medm screens! I've logged in as Ops, myself and Controls. It happens under all of those accounts

10:02 Terra informs me she's about to do some pretty invasive injections

10:44 Injections caused PI to run away and break lock

10:56 put ISC-LOCK into INITIAL_ALIGNMENT to re-align green arms.

Lost more than 5 hour lock 10 minutes before end of shift. SUS OMC and HAM6 ISI tripped. Cause not immediately obvious. Handing over to Ed.

23:00 UTC Jenne is in LVEA by HAM6
23:23 UTC Gerardo back from overfilling CP3
23:27 UTC Disconnected and reconnected conlog-test-master to ~100,000 channels
23:41 UTC Starting initial alignment
23:46 UTC Changed from VERY_WINDY_NOBRSXY to WINDY. Improved X arm transmission and stability.
23:55 UTC Jenne back, whitening AS_A chassis swapped (Jenne, Mark, Filiberto)
00:21 UTC Initial alignment done. Jenne to LVEA.
00:42 UTC Jenne back. Starting relocking.
01:36 UTC NLN
01:55 UTC Jeff K. starting calibration sweeps
03:00 UTC Terra starting excitation of PI mode 2
03:18 UTC Jeff K. done
04:51 UTC Nutsinee to mezzanine to check TCS chiller
04:59 UTC Nutsinee back
05:36 UTC Jenne to LVEA to jiggle RF45 cable
06:01 UTC Jenne back
06:50 UTC Lost lock. SUS OMC and HAM6 ISI tripped.

About an hour ago, I noticed that our sensitivity degraded significantly in the bucket.  This brought our range down to about 20-30 Mpc from the 50+ Mpc we had been at.

Glancing up, I noticed that the RF45 spectrum looks atrocious.  I'm not at all sure what is going on, but it is very bad, and changing with time.

In the attached screenshot, I have the DARM spectrum, the RF 45 and RF 9 spectra, as well as the RF coherences with DARM.  The colors are the same in each plot, with the start times of the data indicated in the DARM legend.  Dark blue is a good time, when our range was good.  Pink and green and red show different times when there was a weird peak in the RF45 spectra.  Brown and light blue are intermediate times when the RF45 is elevated, but not peaky. 

I don't know why this would have anything to do with the new barrel connector that Richard et al. put in earlier today (alog 30375), but maybe I'll go jiggle it.

EDIT:  Before any jiggling, Terra Patrick and I notice that it's steadily getting worse :(