model restarts logged for Thu 03/Nov/2016 -  Wed 02/Nov/2016 No restarts reported

model restarts logged for Tue 01/Nov/2016

2016_11_01 11:14 h1ioppsl0

2016_11_01 11:14 h1psldbb
2016_11_01 11:14 h1pslfss
2016_11_01 11:14 h1psliss
2016_11_01 11:14 h1pslpmc
2016_11_01 11:23 h1calcs
2016_11_01 11:23 h1iopoaf0
2016_11_01 11:23 h1ioppsl0
2016_11_01 11:23 h1ngn
2016_11_01 11:23 h1oaf
2016_11_01 11:23 h1odcmaster
2016_11_01 11:23 h1pemcs
2016_11_01 11:23 h1psliss
2016_11_01 11:23 h1tcscs
2016_11_01 11:24 h1calcs
2016_11_01 11:24 h1iopasc0
2016_11_01 11:24 h1ioplsc0
2016_11_01 11:24 h1iopoaf0
2016_11_01 11:24 h1iopsusb123
2016_11_01 11:24 h1lscaux
2016_11_01 11:24 h1lsc
2016_11_01 11:24 h1ngn
2016_11_01 11:24 h1oaf
2016_11_01 11:24 h1odcmaster
2016_11_01 11:24 h1omc
2016_11_01 11:24 h1pemcs
2016_11_01 11:24 h1psldbb
2016_11_01 11:24 h1pslfss
2016_11_01 11:24 h1pslpmc
2016_11_01 11:24 h1susitmy
2016_11_01 11:24 h1susprocpi
2016_11_01 11:24 h1tcscs
2016_11_01 11:25 h1ascimc
2016_11_01 11:25 h1asc
2016_11_01 11:25 h1hpiham4
2016_11_01 11:25 h1hpiham5
2016_11_01 11:25 h1iopseih45
2016_11_01 11:25 h1iopsush2a
2016_11_01 11:25 h1iopsush2b
2016_11_01 11:25 h1omcpi
2016_11_01 11:25 h1susauxasc0
2016_11_01 11:25 h1susbs
2016_11_01 11:25 h1sushtts
2016_11_01 11:25 h1susim
2016_11_01 11:25 h1susitmpi
2016_11_01 11:25 h1susitmx
2016_11_01 11:25 h1susmc1
2016_11_01 11:25 h1susmc3
2016_11_01 11:25 h1susprm
2016_11_01 11:27 h1hpiham2
2016_11_01 11:27 h1hpiham3
2016_11_01 11:27 h1iopseih23
2016_11_01 11:27 h1iopsush34
2016_11_01 11:27 h1iopsush56
2016_11_01 11:27 h1isiham2
2016_11_01 11:27 h1isiham3
2016_11_01 11:27 h1isiham4
2016_11_01 11:27 h1isiham5
2016_11_01 11:27 h1susmc2
2016_11_01 11:27 h1suspr2
2016_11_01 11:27 h1suspr3
2016_11_01 11:27 h1sussrm
2016_11_01 11:29 h1hpibs
2016_11_01 11:29 h1hpiitmx
2016_11_01 11:29 h1iopseib1
2016_11_01 11:29 h1iopseib2
2016_11_01 11:29 h1iopseib3
2016_11_01 11:29 h1isibs
2016_11_01 11:29 h1isiitmx
2016_11_01 11:29 h1isiitmy
2016_11_01 11:29 h1susomc
2016_11_01 11:29 h1sussr2
2016_11_01 11:29 h1sussr3
2016_11_01 11:30 h1hpiham1
2016_11_01 11:30 h1hpiham6
2016_11_01 11:30 h1hpiitmy
2016_11_01 11:30 h1iopseih16
2016_11_01 11:30 h1isiham6
2016_11_01 12:05 h1alsex
2016_11_01 12:05 h1calex
2016_11_01 12:05 h1iopiscex
2016_11_01 12:05 h1iscex
2016_11_01 12:05 h1pemex

Tuesday maintenance. ADC addition to h1oaf0 caused dolphin crash in corner station. Rebooted h1iscex to check for ADC offset. No DAQ restarts required.

model restarts logged for Mon 31/Oct/2016 - Fri 28/Oct/2016 No restarts reported

SEI: No issues to report
SUS: No issues to report
PSL: Ultrasonic flow sensor installed, not reading back, in contact with vendor
VAC: No issues to report
FAC: Still searching for water leak, 1/2 gallon per minute
CDS: Demod chassis swapped with spare, investigating in lab (alog 31181)

TITLE: 11/04 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: Patrick
SHIFT SUMMARY: Sheila was working hard all night at 36W with a 6 hour lock. Broke lock about 45min ago and Richard is working on possibly broken POP90 demod input.
LOG:

• 14:26 Richard to LVEA pulling chassis for alog31181

I looked over the mechanical drawings of the high power attenuator.  The position of the mount holding
the thin film polarisers is largely set by two dowel pins in the base and the machining tolerances.
However the nominal angle of incidence of the design is 55.014 deg.

    The thin film polariser coating has a measured transmission of 99.552% and 0.036189% for p and s
polarisations respectively, which is a ratio of ~2750:1.  There is no information about the coating's
sensitivity to incident angle but generally speaking these sort of coatings do tend to be a little
sensitive to incident angle.  That said, it might just be that the beam alignment into the high power
polariser is off a little.

FAMIS#6495

No water was added to the crystal or diode chiller, both were at their nominal levels.

I checked that the makeup air in the PSL enclosure was not limiting us at the 20% nominal setting. However, I found that at 100%, it would limit us. Figure 1 shows the difference in DARM between makeup air on/off  at 100% fan speed. Red is makeup air on, Blue, off.  I am particularly concerned with the regions such as between 15 and 22 Hz where the DARM noise increases but the microphone level does not increase. I don’t think we can make accurate estimates of air current noise using the microphones.

Another source of air currents that may produce jitter are the temperature variations along the beam path. Figure 2 shows a FLIR image of the beam path between the HPO and the DBB. I took this image on Monday. The table surface reading was 23 degrees C while the walls of the HPO were 26 degrees and the walls of the DBB were 26.5. I think that the temperature differences with respect to the table may generate an air current down onto the table, crossing the beam path, and up the sides of the boxes. The space between the Front end and the HPO is a similar channel. Downstream of the jitter-reducing PMC, near the main beam path under the periscope, lies the PI piezo controller (Figure 3). Because of its transformer, it is a warm 32 degrees C, nearly ten degrees hotter than the surrounding environment. The cool air coming in to replace the rising current above the controller may cross the main beam path. This would be very easy to move somewhat further away to test for any change in DARM.

for my own edification, I took a look at End X during a wind time on Oct 17. 
wind storm started at about 1 am Pacific time and went for about 8 hours. 

(note to self - repeat this for Oct 14 - see Jim Warner alog
https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=30547)

I can see several things:
1) The wind direction indicator seems to work as long as the wind is blowing hard enough. This is great. 
Conversion is : add (180-37) = 143 degs. 
This makes 90 (towards -Y) into 233 (from the SW, up along the Y arm)
see plots 1&2 for wind history during monitor period.


2) building tilt is still related to wind. From 1 mHz to 20-30mHz, the relation between floor tilt and wind-speed^2 is quite clear - good coherence, time series look similar. 
(plots 3&4)
TF is odd though,(plot 5) it doesn't appear to follow Kramers-Kronig. 


3) The wind speed indicator gives decent data at frequencies below 20-30 mHz. This limit is set by the discrete reporting levels in some ADC.
(see plot 6) There may also be some low pass filters, but those are the biggest limit on data quality. 

• IFO locked at NLN 3:22 and counting
• Robert at EY doing his thing
• ran 3 a2l scripts
• had some success damping a couple of violin modes
• PI not causing trouble tonight. Some SDF issues have been resolved regarding this
• wind is ≤ 10mph
• µSei appears to be trending down a little with it's median being right around the 90%ile

Here are some noise estimates for our current configuration (at 25W).  The PMC PZT noise was measured before tuesdays fix, if I can get some time I will remeasure that tonight.

 These curves are a combination of excess power measurements, gwinc curves for thermal and residual gas, and Evan Hall's noise budget code. 

Obvious things that are still missing are frequency noise and ESD DAC noise.

There are now THREE a2l scripts that we will be running once during EACH lockstretch. Patrick mentioned them in his time log. I'm tagging Opsinfo with the information again.

cd /opt/rtcds/userapps/release/isc/common/scripts/decoup

./a2l_min_LHO.py

./a2l_min_PR2.py

./a2l_min_PR3.py

that is all :)

Over the last couple of days we noticed the POPAIR_B_90 was randonly jumping up and down.

Further investigating this, we found that just touching the SMA connector of the 90MHz LO of the POPAIR_B_90 demodulator box, the phase jumps by easily 90deg.

Smells like that LO input is broken - but we'd have to take the chassis out to fix it.

In preparation for Robert's incursion into the VEA for testing.

J. Kissel, D. Tuyenbayev

We're still not getting the IFO duty cycle to get the desired uncertainty on the single-frequency actuation strength scale factor measurement of the UIM and PUM stage and we're running out of time, so I've increased the SNR of these temporary lines by another factor of ~3 over yesterday's increase (see LHO aLOG 31108).

Oscillator       Freq (Hz)      Old Amp (ct)   New Amp (ct)
ETMY UIM          33.7           180            500
ETMY PUM          34.7            81            300

Hopefully Robert's activities tonight won't impact the ~30 Hz region of the sensitivity, and we can turn things off soon.

Attached is a 60 day trend of PT140 which is a one of the new Inficon BPG402s?  IP7 and IP8 have been a steady 5000 volts for this time period.  Is this a gauge thing?  I haven't been intimate with what Gerardo, John and Chandra have learned regarding the behavior of these new wide-range Bayard-Alpert/Pirani hybrids but this slope looks "not insignificant"

J. Kissel, M. Evans, D. Barker, H. Radkins

A confusing bit of settings wrangling* after the unplanned corner station computer restarts on Tuesday (LHO aLOG 31075) in the SUSITMPI model meant that a large fraction of the EPICs records in the ITM PI system were wrong. As such, we believe this was the cause of battles with Mode 27's PI a few nights ago (LHO aLOG 31111).

In order to fix the problem, we used the hourly burt backups in
/ligo/cds/lho/h1/burt/yyyy/mm/dd/hh:mm/
to restore settings all settings to Monday (2016/10/31), before the computer restarts. 

Further, Matt performed a few spot checks on the system, and suspected it good.

*Settings wrangling:
There were several compounding problems with the SDF system which meant that 
(1) The front-end did not use the safe.snap file upon reboot, and restored bogus values
(2) The safe.snap file, which we'd thought had been kept up to date, had not been so since May.

Why?
(2) The safe.snap file for SUSITMPI used upon restart,
/opt/rtcds/lho/h1/target/h1susitmpi/h1susitmpiepics/burt/safe.snap,
is a softlink to
/opt/rtcds/userapps/release/sus/h1/burtfiles/h1susitmpi_safe.snap.
Unfortunately, *only* that model's safe.snap that had its permissions mistakenly set to a single user (coincidentally me, because I was the one who'd created the softlink from the target directory to the userapps repo.), and *not* the controls working group. That means Terra Hardwick, who had been custodian of the settings for this system, was not able to write to this file, and the settings to be restored upon computer reboot had not been updated since May 2016. Unfortunately, the only way to find out that this didn't work is to look in the log file, which lives in
/opt/rtcds/lho/h1/log/${modelname}/ioc.log
and none of us (save Dave) remember this file existed, let along looked at it before yesterday **.
There are other files made (as described in Hugh's LHO aLOG 31163), but those files are not used by the front-end upon reboot.

I've since fixed the permissions on this file, and we can now confirm that anyone can write to this file (i.e. accept & confirm DIFFs). We've also confirmed that there are no other safe.snap files that have their write permissions incorrectly restricted to a single user.

** Even worse, it looks like there's a bug in the log system -- even when we confirm that we have written to the file, the log reports a failure, e.g. 
    ***************************************************
    Wed Nov  2 16:39:10 2016 
    Save TABLE as SDF: /opt/rtcds/lho/h1/target/h1susitmpi/h1susitmpiepics/burt/safe_161102_163910.snap
    ***************************************************
    Wed Nov  2 16:39:10 2016
    ERROR Unable to set group-write on /opt/rtcds/lho/h1/target/h1susitmpi/h1susitmpiepics/burt/safe.snap - Operation not permitted
    ***************************************************
    Wed Nov  2 16:39:10 2016
    FAILED FILE SAVE /opt/rtcds/lho/h1/target/h1susitmpi/h1susitmpiepics/burt/safe.snap
    ***************************************************


(1) This is quite alarming. Dave has raised an FRS ticket (see LHO aLOG 6588) and fears it may be an RCG bug. I wish I could give you mode information on this, but I just don't know it.



In summary, we believe the issues with SUSITMPI have been resolved, but there's a good bit of scary stuff left in the SDF system. We'll be working with the CDS team to find a path forward.