A Guardian top node as been added to H1: IFO:

(The node should now be visible in the GUARD_OVERVIEW MEDM screen.)

ifo: H1
name: IFO
CA prefix:
module:
  /opt/rtcds/userapps/release/sys/common/guardian/IFO.py
usercode:
  /opt/rtcds/userapps/release/sys/h1/guardian/IFO_NODE_LIST.py
nominal state: ALL_NODES_OK
initial request: ALL_NODES_OK
states (*=requestable):
  50 * ALL_NODES_OK
  20   WAITING_FOR_NODES_OK
  10   NODE_FAULT
   0   INIT

The node is monitor only; it only monitors the status of all other nodes in the system.  This is based on the system previously deployed at L1.

The sole purpose of this node is to report on the full status of the guardian system.  This is done via the node OK channel, and is defined similarly to L1:

• H1:GRD_IFO_OK == 1 ('True' ) - interferometer is observation-ready
• H1:GRD_IFO_OK == 1 ('True' ) - interferometer is NOT observation-ready
• record missing: IFO top node is not running

The check is essentially that all nodes in the system are themselves reporting OK == True, e.g.:

self['OK'] = self['OP'] == 'EXEC' 
             and 
             self['MODE'] in ['AUTO', 'MANAGED'] 
             and 
             self['REQUEST'] == self['NOMINAL'] 
             and 
             self['STATE'] == self['NOMINAL'] 
             and 
             self['STATUS'] == 'DONE' 
             and 
             not self['ERROR'] 
             and 
             self['CONNECT'] == 'OK'

In other words, all nodes are running as expected, are not in error, and their STATE and REQUEST are equal to the NOMINAL state.

The list of nodes being monitored is currently stored at:

/opt/rtcds/userapps/release/sys/h1/guardian/IFO_NODE_LIST.py

The list currently includes:

• ISC_LOCK
• ISC_DRMI    <----  NO NOMINAL STATE DEFINED
• ALIGN_IFO   <----- NO NOMINAL STATE DEFINED
• OMC_LOCK
• ALS_*
• SUS_*
• SEI_*
• HPI_*
• ISI_*

NOTE: the nodes without NOMINAL states defined will prevent the IFO node from ever becoming OK.  We need to either define NOMINAL states for these nodes, or temporarily remove them from the list of monitored nodes.

[Stefan, Jenne]

Earlier in the day, we were struggling to get through the INCREASE_POWER state in the ISC guardian, which happens after the transition of DARM to DC readout.  

After much tracking and searching, we discovered that the problem was that the setpoint for the OMC transmitted power was done as an ezcaread after the PSL rotation stage had already started moving.  Stefan had seen in the past that this kind of system will often have some lag (you're not reading the current OMC DC PD value infinitely fast, so you're constantly changing your setpoint) which causes the system to run away.  

We have changed this to a hard-coded value (defined as lscparams.omc_dcpd_sum_target), so that the DARM offset is changed while the power is increased to keep the OMC DC PD at this value (currently 20 mW).

This seems to now work exactly as we expect, and we're easily able to get past this state.

J. Kissel, E. Merilh, J. Warner, B. Weaver

As we begin to figure out what it means to be on the relocking team, we've made our best attempt at organizing / coordinating all planned maintenance day activities such that we understand their impact on the IFO and can recover from them as quickly as possible. See below. Have patience while we figure this "new" "system" out with you!

All tasks have been arranged and coordinated so as to not conflict with one another. All tasks and estimated times for completion will be added to the reservation system when they are scheduled, and after the task manager has checked in with the operator. PLEASE PAY ATTENTION TO THE RESERVATION SYSTEM (to help, we're going to put it on the big projector during maintanence). As always, please keep the operators informed of your activities as accurately as possible / reasonable throughout this maintenance day so the reservation list can be adjusted accurately. We appreciate your cooperation!

Maintenance Day Timeline:

• SDF system for affected front end computers cleared out (Operators on Shift, Re-Locking Team)
• EX/EY SEI/SUS systems brought to OFFLINE / SAFE via gaurdian (Operators on Shift, Re-Locking Team)
• IMC brought to OFFLINE via guardian (though check to see if bringing HAM2/HAM3 SEI to damped does it for you!) (Operators on Shift, Re-Locking Team)
• CS SEI systems brought to OFFLINE via guardian (Operators on Shift, Re-Locking Team)

• PSL - Periscope Alignment (to be done immediately following PMC/FSS Re-alignment. ET ~4hrs (R. Schofield)
• PEM CAL - This will be going on in multiple places in VEAs. Sensor correction will be turned off during these activities -  ET ~2hrs (V. Roma, J. Palamos)

I ran my brute force coherence script on last night lock. The results are available here:

https://ldas-jobs.ligo.caltech.edu/~gabriele.vajente/bruco_1120811417/

I'll post as summary soon.

Since the IFO has been nicely locking I updated the SUS drfit mon to capture the new ETMx and TMSx alignments.  The time I used to update was 1120863300.

Betsy, Jeff, Ed, Sheila, Kiwamu, JimW, probably others

Because we wanted to reduce the opportunities for chaos tomorrow, we worked on making sure SDF's were clean at the end stations. This involved a lot of asking questions, head-scratching and saying "yeah, it's probably fine".  There are still a number of red tables for the corner station, such as windy blends on the CS BSC's and a bunch of LSC and ASC diffs, but the plan currently only includes restarting end station models.

Rediscovering why we fixed the large ion pump voltages at 7000V in the past -> Changed IP1-6 to 7000V fixed voltage

I have been working on putting together a power dudget for Hanford IFO. I have calulcated the power on the beamsplitter using abolute power on vaious photodiodes, and put this into a shot noise curve model.  I have compared this to shot curve is measured using DCPD null readout.The shot noise curve is taken from the GWinc model.  The parameter file I am using is attached.  These files are available in /ligo/home/eleanor.king/PowerBudget. 

I calculated the power on the beamsplitter using the TR QPDS  (TR_X/Y_QPD_A/B) to determine the power in the arms,  and using the POP sensors(POP_A_QPD, POP_B_QPD POP_A_LF).  The resuts are summarized in the table below.  There is a matlab script with my actual calculations in /ligo/home/eleanor.king/PowerBudget/PowerOnBS.  The recycling gain for the TR is larger than that measured with the POP_PDs.  If calibrate the POP_A PDs to a single shot, same result. I am assuming the TR QPDs are correct.  The resylsing gain calculated by the absolute powers on these PDs agrees with the recycling gain calculated by the relative power change before and after locking using both TR and POP photodiodes.

I have taken an average value of all of the TR QPDs, which is 41(+/-15%).  I have also included lines showing +/- 1 standard deviation in the plot of the shot noise curve.  The photodecetctor quantum effieciency is 85%, and the losses in the arms are 120ppm, measured alog 16579.  Next I plan get a better understanding of the mode matching numbers used for generating the shot noise curve.  (Mode matching into arms and mode matching into SRC, which is currently assume dto be perfect.)

-------

Additional Comments:

Propagation of arm power to recycling gain:

Assume losses in arms 0f 120ppm, alog 16579.

Recycling gain from power on the beamsplitter:

Pinput=IMC_input_power*0.88*Tprm.  (It is power on the beamsplitter that I am calculateing from the photodiodes and putting into the GWINC noise model.  But I find it easier to convert from this to recycling gain, and think in terms of recycling gain.

Some comments on the current photodiode calibrations:

POP LSC Photodiodes were calibrated by Kiwamu in alog 13905, based on the transimpedence of this photdiode.  [cnts/W] = 0.76 [A/W] x 200 [Ohm] x 2¹⁶ / 40 [cnts/V]

TR_QPDs and  POP_QPD calibrated using Dan's calibration alog 15432.  Note the whitening gain changes on these during full lock, so it is important to keep track of the multiple dewhitining filter banks.

The disk drive in control room computer opsws6 has died.  The computer was removed and will be taken in for service, a substitute has been installed in it's place.  The name of the substitute is lveaws2, so don't be surprised when you log in if it has a different name.  The opsws6 computer will be reinstalled when it has been repaired.

(Not Dan)

An accidental INDENTING change in the ISC_LOCK guardian affected a "return True" statement. Result:

- CARM_ON_TR returned true without actually ramping H1:LSC-REFL_SERVO_IN2GAIN to -32dB.
- This resulted in random lock losses later on in the lock sequence.
- This resulted in 6h of head scratching of the assembled commissioning team.

There is a long an ongoing discussion about the python indentation convention. No need to state which side the author of this log is on.

We've been stably locked with a recycling gain of 38 at 23.3 Watts for almost 2 hours (I intentionally broke the lock).   There were some of the slow (20 second) oscillations that cause some of our locklosses durring ER7.  

When I first came in I had some trouble with locklosses durring ENGAGE_ASC.  One problem was just a typo, the other one seems to be when the gain is increased for the SRC2 loop.  I think this caused the increase in POP90 we were seeing last night, which went away.  This morning it kept increasing until the lockl dropped. I moved the transmon QPD offsets back to the pre-vent offsets, powered up to 18 Watts, turned off the ITM loops and adjusted the ITMY alingments slightly to get a decent recycling gain.    The green alignment in the arm was OK here, and the camera positions not off by more than 4 counts so I did not update the references. 

The offsets where we are stable with recycling gain of 38 and 23.3 Watts are:  

Sheila, Evan

Bat control is among the most challenging of control problems. In this case, some patience and a cardboard box were all that was needed to solve the problem.

Evan, Stefan

When we came in the winds were relatively high, so we decided to take another loock at the AS_A_36 phasing. We did so in DRMI.

- To take out the effect of WFS centering and the otherASC loops, we lowered the total ASC gain by a factor 10, and increased the WFS centering gain from 1 to 200 (raising the centering gain by a factor of 20).
- We then drove the SRM at 0.3Hz in PIT and YAW.
- Interestingly PIT and YAW totaly disagree on the phase, by about 125deg...
- Attached are screen shots for the old phases (picture 1), phased for PIT (picture 2), and phased for YAW (picture 3)
- Additionally, while all 4 quadrants show a reasonable signal for PIT, the YAW signal is weak in quadrant 1 and not present in quadrant 2
- We decided to make a pragmatic choice:
  - For PIT AS_B_36_I is already a fine signal - only for YAW it doesn't work
  - Thus we decided to phase AS_A_36_I for YAW...
  - ... and only use quadrants 3 and 4 (the lower two quadrants).
  - Quadrants 3 & 4 also have a similar strength signal (with opposite sign) and similar RF offset (with the same sign)
  - So: this new YAW signal AS_A_36_I should be fine now.

YAW Input matrix for I and Q now is:
H1:ASC-AS_A_RF36_I_MTRX_2_1    0
H1:ASC-AS_A_RF36_I_MTRX_2_2    0
H1:ASC-AS_A_RF36_I_MTRX_2_3    -2
H1:ASC-AS_A_RF36_I_MTRX_2_4    2

With this new YAW sensor we updated the ASC sensing matrix (pictures 3 and 4).

This sensor seemed to do a good job at 17W and at 24W. However at 24W we still saw a 0.4Hz instability - see next alog.

We've noticed two more examples of epics channels freezing for a few seconds.  Yesterday (July 9 16:25:09 UTC) and tonight (0:22:49 UTC July 11)

Earlier today, I wrote a couple out of loop feedforward filters to the BS ISI foton file using Foton. When I hit the load coefficients button (while the ISI was isolated, the ff paths were off, so it shouldn't have done anything), the ISI tripped, hard. It rang up the T240's pretty bad and I couldn't isolate the ISI for several minutes after. Worried I had inadvertantly written some other filter I ran a diff on the most recent archived file and the file created yesterday when Jeff restarted the models. This showed a whole bunch of filter coefficient differences, which shouldn't have been there (as reported by a diff of the two archive files, I don't know exactly what changed, see attached). Talking to Jim, Dave and Jeff, it sounds like the glitch was probably caused by my having used Quack recently (June 22nd) to load some blend filters. Jeff's model restart (and even a prior model restart on June 30) simply inherited that quack-written file. Today was the first time the BS's foton file was opened and saved in Foton. Quack can apparently load coefficients with higher precision than Foton will accept, so when you open and save a "too high" precision filter with Foton, it rounds the coefficients off. Sudden change in precision of SOS coefficients in the blend filters = bad for isolation loops = bad trip.

We've seen this Foton vs. Quack Rounding problem before -- see e.g. https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=3553 -- and it's still biting us.

This sounds like a relatively easy thing to control for, I can think of two ways:

- getting Quack to check and do the rounding on it's own before writing to the Foton file.

- have the post-build script run a "foton -c" on all filter files before the model gets restarted.

Is there someone in the CDS group who can fix this? Maybe it has been? There are several versions of Quack running around, June was my first attempt with it, maybe I used the wrong one.

I used /ligo/svncommon/SeiSVN/seismic/Common/MatlabTools/autoquack.m

 https://svn.ligo.caltech.edu/svn/seismic/Common/MatlabTools/autoquack.m

Last Changed Author: brian.lantz@LIGO.ORG
Last Changed Rev: 7939
Last Changed Date: 2014-02-14 15:38:15 -0800 (Fri, 14 Feb 2014)
Text Last Updated: 2014-02-14 15:48:16 -0800 (Fri, 14 Feb 2014)

Evan, Kiwamu, Jenne, Stefan

- DRMI alignment is back to the old-good one: strategy: Used old slider values for everything but large optics. Tweaked SR3 (for instance) to get the beam spot centred on ASPD. Aligned PRX using PRM and PR2.
- DRMI ASC worked except PRC2 loop (didn't further investigate because we didn't care without the arms)
- Then we focused on MICH freeze:
  - We fine-tweaked the transfer function using a zpk([0.03],[0.054],1,"n")gain(0.555556) filter.
  - This made the gain roughly 1 below 0.1Hz. Plot 1 shows that - if measured coherently - we win up to a factor of 10 reduction at 0.01Hz. (Blue: no MICH freeze, red: MICH freeze)
  - In terms of RMS reduction (position) of the power spectrum, we gain a factor of 2, at the cost of significant noise injection at 8Hz. (Plot 2)

Interestingly, this RMS is now small enough that we spend most of the time in about 1/3 for the whole simple Michelson fringe. Unfortunately there is still slow drift, so parking at a "good" position isn't quite possible. But we are definitely in a regime where simple "fringe velocity" isn't a good parameter by itself. Fringe position must matter too. In our brief attempt to see locking performance changes we didn't notice anything significant though.

However, the next time we have high winds, we should definitely re-evaluate MICH freeze.