Aidan, Terra

A continuation of alog 30522

Previously we'd found that frequency drift over a lock stretch is mode dependent - differential drumhead mode drifts more than that same optic's drumhead mode - but surmised that the ratio of (change of frequency / frequency) between two modes would be constant. However, this assumed a spacially uniform change in temperature --- f_m = g(T) --- when a more realistic assumption is some radial dependence of T, so: 

f_m = g(T(r)), where r is radial vector

Since the energy distribution of a given mode is also dependent on r, we can expect some modally unique self heating response. In other words, the more overlap between a given mechanical mode and the hot spots of the optic, the larger the frequency drift we'd expect that mode to have. 

I've looked at ETMY 15009 Hz diff drumhead mode shift compared with ETMY 8158 Hz drumhead shift during twelve hours of a recent lock. 15009 Hz relative frequency change is larger than drumhead relative change and the ratio begins to level off towards a constant after about six hours. We expect (and model) the opposite - the drumhead mode would have more thermal overlap and thus would shift more. This assumes a mostly centered beam spot (or equivalently, assumes r = 0 at the center of the optic). Looking at Jenne's recent beam spot investigation, ETMY spot position was fairly centered during this time, off in yaw by a few mm. The 15009 Hz mode is differential in pitch. (I still suspect some dependence on torsional vs. longitudinal mode movement as suggested in alog 30522.)

We've started having operators run a2l more regularly and before and after powerup at times to get assurance of beam spot position. 

TITLE: 11/18 Owl Shift: 08:00-16:00 UTC (00:00-08:00 PST), all times posted in UTC

STATE of H1: OBSERVING
OUTGOING OPERATOR: Corey
QUICK SUMMARY: H1 has been locked 16 hours and counting. Sheila sent me an instruction and asked me to play with PRM. I'm waiting for LLO to be out of Observing before doing that. If that doesn't happen I will do it towards the end of shift.

After getting wack values for PI mode ring ups (many orders of magnitude off from expected) earlier in the week, I've refit ring ups and taken new ones and gotten much more reasonable values (I wasn't looking at long enough time stretches to get accurate ring up data before). Note that we haven't had any instability in Mode3 in many days so I haven't been able to remeasure it. 

Mode 3 hasn't been unstable in many days and Modes 27 & 28 are only unstable during the initial ~ 1-2 hours of transient. Attached are two 30 hour stretches of the damping output of the three modes during recent long locks (Mode3 had no output so I left off) and the simulated HOOM spacing to get an idea of when the modes are ringing up enough to engage damping loops. Left is a few days ago and right is the current lock. Note that Mode26 looks continuously unstable during the 11/16 lock, but it could be that the damping gain is triggering below an actual unstable amplitude; compare to the current lock where we set the gain for Mode26 to zero just after 19:30 (so there would be no damping output) but it has remained stable and low since then with no need to damp. 

Operators and myself are currently turning off gain and measuring ring ups during different times of lock stretches to get gains at different stages of the thermal transient. 

TITLE: 11/18 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Commissioning
INCOMING OPERATOR: Nutsinee
SHIFT SUMMARY:
Did not get to ANY of my ToDoList!  Because H1 has been locked the whole shift (going on 14hrs).  So I passed these items on to Nutsinee.  Nutsinee did notice big Violin Modes.  I talked to the LLO Operator to check their status (Robert still doing PEM injections), so Nutsinee will try to damp the violins (because if they get rung up more it will make locking difficult.).  LLO will contact us when they go to OBSERVING.

LOG:

• 0:29 Out of Observing for H1 ASC work (Sheila)
• 3:09 Calibration work starts (Kissel & Crew)
• 5:18 OBSERVING!

To Do:

• Load Coefficients for PR3 & SR3 (they've been RED on CDS Overview).
• A2Ls at DC ReadOut & NLN (for Terra)
• PI settings changes for Mode 26 & 3 (for Terra)

The only Oplev worth note is HAM2 pit/yaw which is well over +/-10urad.

This closes FAMIS #4702.

Jim's new WEEKLIES medm is pretty cool!

TITLE: 11/17 EVE Shift: 0:00-08:00 UTC (16:00-0:00 PST), all times posted in UTC

STATE of H1: OBSERVING
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    Wind:  under 5mph
    Primary useism:  Trending over last 24hrs
    Secondary useism:  --
QUICK SUMMARY:

Handed an H1 in Observing (9+hrs).  Taken out of Observing for Sheila to improve PRC alignment (related to the Allowing the Refl WFS & Beam Diverter Closing steps in Guardian).  H1 has now been transitioned to calibration for the next few hours (Kissel).

I do have a To Do List for when we drop out of lock:

• Load Coefficients for PR3 & SR3 (they've been RED on CDS Overview).
• A2Ls at DC ReadOut & NLN (for Terra)
• PI settings change (for Terra)

This fan has been on continuously for the past several years.  I noticed that it is starting to get noisy - mostly due to an unexplained increase in flow.  Even at the minimal speed setting, the flow seems to be excessive for some reason.  This fan exhausts the area above VBOA in Room 169, as well as, the two air-bake ovens in the adjacent Vacuum Prep Lab -> I will add signage to the air bake ovens so as to notify would be users that the exhaust is off and describe the details for turning it back on.  

SudarshanK, DarkhanT, TravisS, RickS

Yend calibration measurements were performed on 10/31/16 using the WS1 working standard.  The results of those measurements were used to generate the calibrations for the Pcal Rx and Tx power sensors.

On 11/9/16 we repeated the measurements using a new working standard, WS3, because WS1 was accidentally dropped on the concrete floor, changing it's responsivity.  We decided to retire it and replace it with WS3.  The time series for these measurements revealed that the peak-peak variations were sevaral times higher than usual.  The power sensor calibrations differed from the earlier measurements by as much as 0.5%.  The power source to which the WS electronics were connected was suspected to be the cause of the excess noise.  

On 11/15/16 (two days ago) we repeated the measurements with WS3 but with the WS electronics plugged into a different power source.  The signal variations returned to the expected values so it seems that the power source was the issue.

The new (previous) calibrations (N/V) are:

Tx: 1.5171e-9 (1.5160e-9) N/V (0.07% difference)

Rx: 1.0497e-9 (1.0475e-9) N/V (0.21% difference)

We will continue to use the "previous" values for the signal calibrations and make periodic checks during the oberving run at about 2-month intervals, as for all the end stations.

With full lock, the H1:LSC-Y_TIDAL_REDTRIG_INMON has a value of 175000.  The H1:LSC-Y_TIDAL_REDTRIG_THRESH_OFF had a value of 10 and it takes almost 4 seconds to reach that threshold.  Meanwhile, the ISC was still driving HEPI and does so quickly causing the ISI T240s to be driven to trip.

In the attached 10 seconds trend, the lock loss is clear with the ASC TR SUM dropping along with the REDTRIG power.  The signal finally reaches 10 almost 4 seconds later seen on the TIDAL_CMD trace.  Meanwhile, ISC has rapidly driven HEPI >100um, quite quickly.  By then the T240s are too far gone (Red Trace #1) and will trip the WD.

Sheila & Daniel have increased the ON/OFF thresholds to 8000/5000.  This should work for 2 watt lock and reduce our delay to under two seconds--maybe enough.  But why is this suddenly a problem?  Or is it some series of unfortunate events that made it happen this time?  Scaling the thresholds with power is the better albeit complex solution.

TITLE: 11/17 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 64.0999Mpc
INCOMING OPERATOR: Corey
SHIFT SUMMARY: Lockloss in the early part of my shift, it tripped HAM6, ETMY ISI, and then 7min later HAM3. After ITMY stopped swinging, it was straight foward to bring back up. Since then it has been locked for 6hrs in and out of Observing for me to run a2L and Terra will sometimes turn on/off PI gains.
LOG:

• 17:20 Lockloss. Tripped HAM6 ISI and ETMY ISI. Then 7min later HAM3, earthquake maybe but nothing on the BLRMS yet.
• 17:28 Chandra to LVEA
• 17:50 Chandra out
• 18:08 Joe heading out to Yarm to do more beam tube enclosure sealing
• 18:12 Richard heading to EY roof to replace anemometers checking phones in building, then to MY to roof for anemometers
• 18:18 Observing
• 18:36 Fil to EY to help Richard
• 19:15 Ken will be hammerdrilling in the mech room. Might be worth trying to see if we can see it.
• 20:02 Out of Observing to run a2L
• 20:09 Observing
• 20:15 Joe back
• 20:19 Ken is done hammerdrilling, now headed to the mezzanine
• 20:33 Richard to MX for anemometer replace
• 21:28 Out of Observing to run a2L
• 21:33 Observing
• 21:37 Richard back
• 21:51 Ken all done
• 22:57 Out of Observing to run a2L
• 23:19 Observing. Should have been for the last 10min but I was distracted by other work.

I made a few small tools that others may want to use. Please let me know if you have any questions, bugs, or complaints.

(userapps)/guardian/grd_grep.bsh - This allows you to grep through the files of a specified Guardian node.

grd_grep.bash - Grep through the files of a particular Guardian node
Usage: grd_grep <NODE_NAME | -h | --help> <GREP_ARGS>
Where:
    NODE_NAME (required) Is the caps name of the Guardian node (Ex: ISC_LOCK)
    <-h | --help>             Print this usage
    GREP_ARGS (required) are all of the normal arguments used with grep.
        By default, it is ran with -n
        (Ex: [grep] -ni "Test String"
 
 
Examples:
    $ grd_grep.bsh DIAG_MAIN -i "SEISMIC"
            or
    $ grd_grep.bsh ISC_LOCK "SUS-ETMY_M0_LOCK_L"

I just aliased this as "grdgrep" and it becomes very easy to find the lines of wanted code in one or more files of that node.

(userapps)/cds/h1/scripts/ws_timer.bsh - A simple timer that will ding when complete.

ws_timer: A simple timer for control room use. Ctrl-z to stop.
Usage: wstimer [seconds] [(-h|-m|-t) arg]
 
 [seconds]      Set a timer for that number of seconds
 -h|--help      Print this help menu
 -m|--miniute   Followed by an int will set a timer for that many minutes
 -t|--time      Followed by a time will ding at time. Times are local and
                    in the form of "hh:mm"

This ended up being more useful that I thought. Alias as just "timer" and it is a great tool while in the chair.

Replaced and verified direction of both Mid Station Anemometers. 

model restarts logged for Wed 16/Nov/2016 No restarts reported

DAQ: fw2 and fw0 are running framecpp_2.5.2, fw1 still running older version. Test code getting and comparing framed data from nds0 and nds1 showing no differences over past 30 hours. LDAS also not reporting any data differences between fw0 and fw1 frames.

Beverly

Complete results may be found on the DetChar Wiki.

• ER10 started at 8:00 PST (16:00 UT) on 14 Nov.
• The duty cycle for this DQ shift was only 12% (counting all of Monday UT) with no observing time on Monday, one lock of 1.5 hours (and several shorter ones) on Tuesday, and two locks of 2 and 4 hours respectively on Wednesday.
• The BNS range varied between 60 and 70 Mpc, covering most of this interval in any of the longer locks. There did not appear to be trends.
• The glitching around the 1083 Hz calibration line persists through the DQ shift.
• Broadband noise ranging up to around 100 Hz comes and goes during observation locks. IThe noise may be weaker in the last lock of the shift.
• See alog ( 31524) for a discussion of fixing the IMC WFS switch and updating a control loop for REFL45 for LSC PD saturations. The latter had appeared dark red (rather than green) in this plot. This was noticed on 15 Nov and appeared fixed on 16 Nov. However, REFL135, which was green on 15 Nov was dark red on 16 Nov.
• Bounce and roll modes were prominent in all the locks except the final one.
• It is possible that the instrument stability is improving although the bounce and roll modes are back on 17 Nov.
• Thanks to LHO fellows Ansel and Young-Min for consultation.

I have determined new frame data rates for H1 and L1.  This was last updated before ER9 ( see aLOG 26706 )

I have only run the 'commissioning' rate scripts, as we have no science frames anymore

You can find the scripts (work at either site) at https://llocds.ligo-la.caltech.edu/data/keith.thorne/DataRates
You can find the L1 data at https://llocds.ligo-la.caltech.edu/data/keith.thorne/DataRates/2016-11-17/, the H1 data at https://lhocds.ligo-wa.caltech.edu/exports/keith.thorne/DataRates/2016-11-17/.

I have attached the summary data sheets in PDF and XLS formats as well as channel lists. These are all raw (uncompressed) rates.

Note: The H1 data rates are inflated (by 2.1 MB/s) by 8 64KHz channels from the SUS/OMC PI models.  These 'artificial' channels (set = 0) are no longer needed after RCG 3.1.1 and have been removed from L1.  This should be done on H1.

The raw H1 fast data size has shrunk from 56.19 to 55.20 MB/s (decrease of 1%).
The raw L1 fast data size has shrunk from 41.67 to 40.56 MB/s (decrease of 3%). Mostly removal of 6 64KHz channels.
The raw H1 fast data rates are now 7% larger than L1 fast data rates

From DAQ MEDM screens
H1 frames are ~1670 MB / 64 sec -> 26 MB/s
L1 frames are ~1400 MB / 64 sec -> 20 MB/s

See the Trip plot attached.  We think the LVEA was unoccupied...did the rack get a glitch?

Nothing appears out of the ordinary.  See attached screenshots.  FAMIS 6872.

I've discovered HAM2 ISO_Z and ISO_RZ are well correlated to alignment changes in MC1 pitch and yaw, MC3 yaw, and anticorrelated with PR3 OSEM yaw.

It seems odd that the nm sized ISI ISO signals are showing up in the optic alignments, however this is what I find, and the optic OSEM alignment changes are not small.

I also looked at optical lever signals, and see that HAM2 and PR3 optical levers don't corellate to ISO or OSEM signals long-term.  HAM2 oplev does occasionally see a large ISI excursion, and correlates with multiple ISO DOFs.

Plot 1: MC1, MC3, and HAM2 ISI ISO_Z and ISO_RZ, 400 days

• MC1 pitch, alignemt range of 80urad
• MC1 yaw, alignment range of 30urad
• MC3 ptich, increasing
• MC3 yaw, alignment range of 15urad
• HAM2 ISI ISO_Z
• HAM2 ISI ISO_RZ

Plot 2: PR3 OSEM yaw, oplev yaw, HAM2 oplev yaw, HAM2 ISI ISO_RZ, 400 days

• PR3 OSEM yaw
• PR3 oplev
• PR3 OSEM yaw, alignment range of 85urad
• PR3 oplev yaw is essentially flat (ignoring the large jump/recovery)
• HAM2 oplev long term
• HAM2 ISO_RZ, and I conceed the changes in this signal seem very small at a couple hundred nrad

Plots 3 and 4: HAM2 oplev pitch, HAM2 ISO_RZ, ISO_X, 400 days

• HAM2 oplev pitch does see large ISI excursions, and correlate to ISO_RZ and ISO_X, and not shown, ISO_Z and ISO_Y

Summary:

• HAM2 ISO_Z and ISO_RZ are showing up in HAM2 optic OSEM pitch and yaw signals
• MC1, MC3, and PR3 OSEMs see the ISO changes (likely other HAM2 optics, but I didn't look tonight)
• HAM2 and PR3 oplevs don't track the changes seen in the ISO_Z/RZ or optic OSEMs
• HAM2 oplev in it's current form is not going to provide a diagnostic for the IO chain as I had hoped

I've added a Weeklies screen to the Sitemap to make it easier for the operators to do FAMIS tasks. It can be found under the OPS tab. Pretty simple, there are a number of buttons that launch different scripts for different FAMIS tasks, so operators don't have to navigate from the command line. Operators and detector engineers should feel free to add tasks to this screen, I added the ones I could think of. Hopefully we can rely on this screen to make sure that operators use 1 version of a script, and give cognizant engineers some control over which version is used.

I also encourage anyone with any artistic inclinations to make it look nicer. I got nothin'.