8:08 locked Low Noise

8:36 set to Observing after clearing a bunch of SDF diffs (Betsy had told me to set them to Not Monitored since they are still in the process of creating the observe.snap). Sheila helped clear some that couldn't be Not Monitored due to the 'too many decimal places' issue.

9:30 I noticed that the OMC had a red CFC block on the CDS overview.  The overflow was incrementing rapidly.  I tried to reset it anyways, but to no avail.  I then tried calling several people, none answered.  Finally I called Corey to let him know about the situation.  Eventually, I got a couple of calls back suggesting that I let it ride since we were already in Observing.  Looking at a few trends, I saw that this began during the locking sequence.  Perhaps the CDS overview had a red block in it, but I failed to notice (at least it didn't prevent me from going to Observing). 

10:30 lockloss due to 5.9 EQ in Alaska.

During the ~2.5 hours lock stretch, our range has been steadily trending downward, perhaps related to the OMC issue (?).

Summary

EPICS values used for calculation of DARM time-varying parameters (DCC T1500377) in GDS pipeline and with PCALMON data were updated on 10-Sep-2015,   01:03:38 PDT (10-Sep-2015,   08:03:38 UTC) using H1 DARM model for ER8/O1. We will analyze DARM time-varying parameters (sometimes generalized as "kappas") calculated in GDS pipeline in GDS frames from nominal low-noise lock stretches after this change.

Previously values written in these EPICS records were calculated based on ER7 model (see LHO alog 20452), now we used ER8/O1 model with parameter file "H1DARMparams_1124954397.m", the summary of this param file residuals were given in LHO alog 21318. There is an additional phase of 136.7 degrees that was fudged into H1:CAL-CS_TDEP_REF_INVA_CLGRATIO_TST_(REAL|IMAG) that should provide kappa_tst with a reasonable phase value (ideally the phase of the kappa_tst should be 0), this was done so that we can test overall performance of GDS pipeline when calculating kappas.

These are not the final values for ER8/O1, currently H1 DARM OLGTF model is under testing using different DARM OLGTF and PCAL2DARMTF measurements and possibly will go through minor adjustments.

New values were accepted in SDF_OVERVIEW (however some of the values still does show up in "diffs", that's because SDF_OVERVIEW cannot handle very small values).

Details

Earlier we had a hard time getting reasonable values for kappas, in Maddie's GDS code as well as recent Greg Mendell and Sudarshan's Matlab code using PCALMON data (see LHO alog 21102). One of the reasons why calculations wouldn't fully agree with actual parameters was that EPICS records had values calculated from ER7 model, which we know is not up to date anymore. But I belive that even after we got a fairly good approximated ER8 model there's probably something missing in EPICS actuation coefficient for x_{tst} calibration line. We're still investigating the source of this discrepancy.

The script that extracts values needed for calculation of kappas from DARM OLGTF model was committed to calibration SVN:

CalSVN/aligocalibration/trunk/Runs/ER8/H1/Scripts/CAL_EPICS/writeER8model_TDEP_EPICS.m

Output files (raw epics values and more verbose log) were committed to the same directory, detailed log is also attached to this alog.

The EPICS values correspond to the model file and the parameter file in calibration SVN at rev. 1374:

CalSVN/aligocalibration/trunk/Runs/ER8/H1/Scripts/DARMOLGTFs/H1DARMOLGTFmodel_ER8.m

CalSVN/aligocalibration/trunk/Runs/ER8/H1/Scripts/DARMOLGTFs/H1DARMparams_1124954397.m

This morning Elli and I spent some time on phaseing of AS WFS, and again this evening we picked it up again.  More details to follow, but here are some highlights:

We can use a consistent phase on all 4 quadrants to get reasonable signals.  

In PRMI we can use 40 degrees for all quadrants on ASA36 to get most of the static signal into the I phase, and 90 degrees on AS 36 B.  In this situation the Q phase has a good signal for the BS, with the transfer function from BS pit drive (1000 cnts, 12 Hz) to each quadrant having all the right phase relationships.  

In DRMI 90 degrees still seems like a good setting for keeping the static signals in the I phase for AS B, while AS A it seems like 20 degrees is a better phase, except for quadrant 4 where it seemed like the phase should be 70.  I assume that this could be caused by a misalingment, and should not be a real since all the phases were consistent in PRMI.  

We closed the loops on AS B in DRMI, which worked fine, and locked.  The phases of the transfer function to each quadrant stayed consistent in DRMI, full lock, and at 10 Watts.  I left the excitation on the BS as I powered up, starting around 7:08 UTC sept 10th.  Durring this time all the quadrants on AS 36 A had an R phase of 20 degrees, all the R phases for AS B 36 were 90, and the yaw matrix for both wfs was normal (it included all quadrants.)  After powering up to 10 Watts we saw the usual runaway where POP90 grows.  We can perhaps look at this data to see if our sensing matrix was changing durring some thermal process. 

After several minutes at 10 Watts, I tried to recover a good operating point by moving the offset in AS_C, which did not work and broke the lock.  

Everything is reverted and we are going back to locking with our old arangement for AS WFS.  

LVEA: Laser Hazard
IFO: UnLocked
Intent Bit: Commissioning  

All Times in UTC (PT)

23:30 (16:30) Take over from TJ
23:30 (16:30) Relocking the IFO
23:52 (16:52) Locked at NOMINAL_LOW_NOISE, 22.7W, 68Mpc
00:02 (17:02) Lockloss – commissioning activities
00:36 (17:36) Locked at NOMINAL_LOW_NOISE, 22.7W, 68Mpc
00:38 (17:38) Lockloss – commissioning activities 
01:09 (18:09) Locked at NOMINAL_LOW_NOISE, 23W, 68Mpc
01:10 (18:10) Lockloss – commissioning activities
01:31 (18:31) Locked at NOMINAL_LOW_NOISE, 22.9W, 63Mpc
01:47 (18:47) Set intent bit to Observing, 22.9W, 69Mpc
02:25 (19:25) Set intent bit to Commissioning – Evan working on noise characterization  
03:05 (20:05) Range bump (over 80Mpc) due to calibration work
05:13 (22:13) Lockloss – Commissioning activities
05:30 (22:30) Adjust ISS diffracted power from -2.02v to -2.00v – diffracted power now at 8.4%
 

 Shift Summary & Observations:

- Attend all hands meeting until 23:20 (16:20)
- Evan working on noise characterization 
- Over 80Mcp range bump due to calibration work – NOT REAL
- At the 05:13 Lockloss, the OMC watchdog tripped 4 seconds after the Lockloss
- The first part of the shift was given to the commissioners while LLO was down. After LLO came back up and was in Observing mode, LHO switched to Observing mode as well. We had a 4-hour lock stretch in the middle of the shift with good range, moderate wind, no significant seismic activity. After losing lock, commissioning work continued until the last hour of the shift. Balance of the shift was spent relocking and doing fine tuning commissioning.        

While we were in commisioing mode, we had a strange non stationary spectrum, with large non stationary noise also appearing in MICH and PRCL.  I quickly looked at some triple witness sensors, since we've had issues with these recently, but didn't see anything obvious.  

The strange noise started a few minutes before 2:50:34 UTC Sept 10, seemed to get better for a few minutes and then got worse again around 2:57:21. 

We don't think that this was related to any comissioning we were doing.  Evan had temporarily lowered the DARM ugf and put a notch in, (which should cause anything like this).  He undid both changes when we saw the unusual noise, but the noise was unaffected by his change.  

We aren't going to investigate any further right now since the problem seems to have gone away.  

Last night, I moved the PRM coil driver switching to happen before the CARM offset reduction (21321 ) with the idea that this would make the locklosses it causes less painfull as they would happen earlier in the sequence.  We tried it this way once and things seemed fine, but then Travis later had a hard time locking.  Many of his locklosses were due to PRM coil driver switching.  This morning TJ and I moved this back to the DRMI on POP state (after CARM ofset reduction)  and we also had locklosses caused by it, although we were able to make it to full lock sometimes.  

This afternoon TJ and I looked into this a bit.  We looked at the amplitude of the glitches as seen by noisemon, and they are the same today as they were August 30th.  We also looked at how the PRM dirves respond to this glitch. Two weeks ago, we were nearly saturating PRM M3 each time we switched, in the past few days we have been nearly saturating M1 as well.  This is not suprising with the higher offloading gain intended to help us lock durring high ground motion (alog 21216)

I put an elliptic low pass at 10 Hz in PRM M1, this reduces the glitch in M1 by an order of magnitude.  We've survived the switching 3 times since we did this, and it is in the DRMI guardian.  In the two screenshots you can see an example of the glitch with switching this morning, and the second one shows the glitch with the new low pass engaged.  We are still in danger of saturating M3, so it would still be a good idea to look at tuning the delay if we get a chance.  

Log:

Times in UTC (PST)

• 1500 (800) Travis handing off the IFO not locked. He had issues for the past ~4:30 of his shift.
• 1557 (857) Nominal Low Noise. There was a problem with the PRM switching, commissioners moved it to an earlier state to save time, but this did not work.
• 1701 (1001) JimB to power cycle Trendwriter0
• 1727 (1027) Kyle to EX to unplug aux cart and then plug in ion pump
• 1740 (1040) Trendwriter0 id dead until further notice
• 1756 (1056) Dick G to optics lab
• 1812 (1112) Kyle back
• 2115 (1415) Evan to CER

Summery:

Commissioning or other activities for my entire shift. Not much to report from me.

Hugh and I have switched the SDFs of many of the SEI, SUS, the LSC, and ASC to compare current settings (OBSERVE.snap) to the NOMINAL LOW NOISE LOCK from around 2:30pm (local) today.  It then broke lock so we have to wait for the next lock stretches to finish - likely to be done tomorrow, so bear with us while we get this up and running.  Background:  we are redirecting to use SDF as the main settings configuration control monitor for O1, so we are now working quickly to do some smallish reconfiguring.  This involves setting the SDF such that it will monitor ~all settings of the IFO and will be all-GREEN when the IFO is in NOMINAL LOW NOISE.  This means it wiill have lots of reds after lockloss and during lock acquisition.  (We will work to reincorporate it's feature to be a troubleshooting tool soon.) 

For the next day or so, while in-flux, it may mean that there are some red SDF alarms that will not let you hit the INTENT button.  If this is the case, please feel free to call me and we'll work through the short diff list, or simply set any DIFFs to NOT MON and I will adjust them as needed tomorrow.

In the long run, the idea is that the SDF will monitor all settings, including guardian manipulated ones, except the few things which change between lock stretches (such as VIOLIN mode damping, HPI Setpoints, ISS REF CAV adjustments, and OPTICALIGN slider values).

One possible cause of the DAC overflows we've been seeing (the ETMY saturation warnings) is something at high frequency using up the range of the drive. We only record the last output to the ESD DAC at 2048 Hz, so we don't see anything happening above a kHz. It's possible some high frequency line or feature bumps up against 2^17 counts and that makes the DAC overflow, which then causes a much larger glitch - once there's an overflow the whole thing will go crazy.

To try to investigate this, the simplest thing to do is to record a channel like SUS-ETMY_L3_MASTER_OUT_LL_DQ at the highest rate possible, or something equivalent that monitors the actual signal sent to the ESD DAC. Once an ETMY overflow happens, save the data for several seconds around the time somewhere that people offsite can access it, preferably in a common format like ASCII or frames. Some time without an overflow would be good for comparison. Then we can see if there's something using up the drive range, or if it's just something sudden from somewhere else in the DARM loop. A few instances of overflows would be nice.

If this is not possible, there's another tack we can take but it's more involved, so I'd prefer this.

Similar to what I did yesterday for HEPI re 21309 I made OBSERVE.snap files for all the ISIs.

However, I came upon an issue for the BSC ISIs where toggling the MONITOR SELECT choice button to ALL left the SDF with differences.

These come from the following: When an ISI or HEPI Isolates, it observes and averages the free hanging position for 5 seconds and puts this mean into the SETPOINT_NOW.  This makes the residual error of the loop zero when it is turned on.  After the loop is completely on, the Guardian puts the Reference Location (e.g.: H1:ISI-BS_ST1_CPS_X_TARGET) into the SETPOINT_NOW record causing the Isolation loop to servo to that same position from platform trip to platform trip.  That is for some but not all platforms/dofs.  All HEPIs and all HAM ISIs are resorting the TARGET.  No BSC ISI is restoring the Target location.

So even though I just took a new OBSERVE.snap, many of the BSCs showed diffs out at e-10 to -15 in the SDF.  I'm not sure why that was variable but it brought the realization forward that each time a BSC ISI tripped, the new SETPOINT_NOW would be different (as it would for all the HEPIs and HAM ISI,) and would not be replaced by the TARGET Location at the end of Isolation (as it would be for the HEPIs and HAM ISIs.)

So for the BSC ISIs, after getting the OBSERVE.snap loaded into SDF as done for the HEPIs and HAM ISIs, the NOT MONITORED channels were all set to monitor and then the SETPOINT_NOW channels were set back to NOT MONITORED.  Given this, the BSC ISIs will not be pink and the MONITOR SELECT button will be set to MASK, rather than ALL like the HEPIs and HAM ISIs.  It may be that I will do the same for all the HEPIs and HAM ISIs if that is best for consistency, TBD.

It seems like shifting the pole from 120Hz to 1.2Hz (alog 21300) reduced the beam jitter greatly only at 60 and 180 Hz. The broad jitter noise floor might have been somewhat reduced between 10 and 100Hz, but I see no change at 300Hz. In the attached, references are before, current traces are after.

Robert reported that the coupling to the chiller water flow dominates the periscope motion at around 300Hz (alog 21273).

Laser Status:
SysStat is good
Front End power is 33.02W (should be around 30 W)
Frontend Watch is GREEN
HPO Watch is RED

PMC:
It has been locked 8.0 days, 1.0 hr 19.0 minutes (should be days/weeks)
Reflected power is 2.097Watts and PowerSum = 26.62Watts.

FSS:
It has been locked for 0.0 days 1.0 h and 10.0 min (should be days/weeks)
TPD[V] = 1.498V (min 0.9V)

ISS:
The diffracted power is around 7.045% (should be 5-9%)
Last saturation event was 0.0 days 1.0 hours and 10.0 minutes ago (should be days/weeks)
 

In prep for turning ~all settings configuration control over to SDF, I have turned off the Set Point Monitoring (SPM) feature on the SUS guardians that were in false alarm (yellow).  We will not be using the SPM feature as a requirement for O1, although any ones that go into alarm which are stll enabled may be used as a good troubleshooting tool.  (Most that are enabled have been green and running for many months now, so can't hurt to stay on.)

J. Kissel, K. Izumi

We haven't gotten these two measurements since the tail end of Calibration week, and we want to confirm that our model (that's under development) is still valid after 1.5 weeks, so we've taken new DARM OLG TFs and PCAL to DARM TFs. We'll analyze these in the next few days and confirm what we can determine from the calibration lines.

Data lives here:
/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/ER8/H1/Measurements/
DARMOLGTFs/2015-09-08_H1_DARM_OLGTF_7to1200Hz.xml
PCAL/2015-09-08_PCALY2DARMTF_7to1200Hz.xml

For the record, this pair of measurements takes almost exactly 1 hour. However, to knock down the measurement time to that one hour, one needs to run the DARMOLGTF first, and start the PCAL TF once the DARM OLG TF hits the frequency points around 15 to 10 [Hz]. 

Note, calibration lines were turned OFF during these measurements. They're now back ON.

Collected OBSERVE.snap files for use in full observing configuration monitoring.

With the SDF happy (green) and Guardian Nominal (Robust Isolated) and green, I made an OBSERVE.snap file with SDF_SAVE screen

choosing EPICS DB TO FILE  & SAVE AS == OBSERVE

This saves the current values of all switches and maintains the monitor/not monitor bit into the target area, e.g.   /opt/rtcds/lho/h1/target/h1hpietmy/h1hpietmyepics/burt as OBSERVE.snap

This file is then copied to the svn: /opt/rtcds/userapps/release/hpi/h1/burtfiles as h1hpietmy_OBSERVE.snap and added/commited as needed.

The OBSERVE.snap in the target area is now deleted and a soft link is created for OBSERVE.snap to point to the svn copy:

ln -s /opt/rtcds/userapps/release/hpi/h1/burtfiles/h1hpietmy_OBSERVE.snap OBSERVE.snap

Finally, the SDF_RESTORE screen is used to select the OBSERVE.snap softlink and loaded with the LOAD TABLE button.

Now, for the HEPIs for example, the not monitored channels dealt with by the guardian will be a different value from the safe.snap but, the not monitored channels are still not monitored so the SDF remains green and happy.  And if the HEPI platform trips, it will still be happy and green because, the not monitored channels are still not monitored.

What's the use of all this you say?  Okay, I say, go to the SDF_TABLE screen and switch the MONITOR SELECT choice to ALL (vs MASK.)  Now, the not monitored channel bit flag is ignored and all records are monitored and differences (ISO filters when the platform is tripped for example) will show in the DIFF list until guardian has the platform back to nominal.

Notice too that the SDF_OVERVIEW has the pink light indicating monitor ALL is set.  This should stay this way unless Guardian is having trouble reisolating the platform and then the operator may want to reenable the bit mask to make more evident any switches that guardian isn't touching more apparent.

Evan, Sheila, Jeff B Ed

Earlier today we were knocked out of lock by a 5.2 in New Zealand, the kind of thing that we would like to be able to ride out.  The lockloss plot is attached, we saturated M3 of the SRM before the lockloss and PRM M3 was also close to saturating.  

While LLO was down, we spent a little time on the offloading, basically the changes described in alog 21084.  This offloading scheme worked fine in full lock for PRM, however we ran into trouble using it durring the acquisiton sequence.  Twice we lost lock on the transition to DRMI, and twice we lost lock when the PRM coil state swtiched in DRMI on POP.  Hpwever, we can acquire lock with the new filter in the top stage of PRM and SRM, but the old low gain (-0.02).  We've been able to turn the gain up by a factor of 2 in full lock twice, so I've left the guardian so that it will turn of the gain in M2 (before the intergrator) in the noise tunings step. 

If anyone decides they need to undo this change overnight, they can comment out lines 344-347 and  2508-2514 of the ISC_LOCK guardian. 

Before we started this, the PRM top mass damping was using 10000 cnts rms at frequencies above a few hundred Hz, because of problems in the OSEMS (alog 21060 ).  Evan put some low passes at 200 Hz in RT and SD OSEMINF which reduces this to 2000 cnts rms. Jeff B accepted this change in SDF.  

 The second attached screenshot shows the PRM drives, the references are in the minutes before the earthquake dropped us out of lock.  The red and blue curves show the current drives, with the high frequency reduction in M1 due to Evan's low pass, and the new offloading on.  The last attached screenshot shows SRM drives with the new offloading.  

• Coil drivers all in low noise state.  Now all suspensions will be in their low noise state once the guardian has completed.
• We tried turning off ITM opLev damping, which eventually caused an alignment instability. 
• SR3 OpLev damping has been off all day, but we seem to be stable at high power without it. 
• We tried reducing the ETMY ESD bias by 33%, and increased the gain in drivealign.  
• Hang did some A2L tuning, and saw that the coefficients had changed. 
• Matt and I looked at Kiwamu's calibration line, and used an rms monitor of it to try to adjust the OMC QPD offsets to maximise the optical gain.  We changed the offsets slightly, but we were mostly only able to decrease the optical gain.  We noticed that the optical gain is moduated by some low frequency (microseism).  The seismic noise has been verry low.  
• There is some MICH coherence between 40-80Hz, we can revisit MICH FF tomorrow. 
• We measured CHARD PIT A2L, we would project that this is a factor of 2 below DARM at 20 Hz, and a factor of 5 below at 50 Hz.  This was with the low pass off, before Stefan's work.
• PI monitor is running for the Y arm.
• Evan updated some calibration, alog on the way tomorow. 
• Evan did not hit load on all the unmodified filters in various models, since the IFO is locked.  
• Intent bit is undisturbed.

None of these things changed the DARM noise.  (just the calibration)