The Problem: The SEI configuration nodes have been in error since Friday at 2:58 local due to a bug in the WD decorators, and a bit of oversight by the one who coded that part (me). The regular SEI guardian nodes were tripped and then someone requested the config nodes to change, but these config nodes were not designed to be changed when everything was tripped. The decorator that caused the isse is just copied code and so it tried to jump to a state that didn't exist. Glad this was found before O2.

The Fix: removed all of the decorators that check on WD's, isolation, and damping of the regular nodes. Jim W and I talked and decided that it wasn't really necessary for the config nodes to even have to watch that, so lets lose them. Changing the config states while it is tripped or isolating was thought to maybe be an issue before, but doesnt seem to pose a problem anymore.

• top left - SEI_HAM6 guardian manager - PAUSED
• middle left - ISI_HAM6 guardian - HIGH_ISOLATED now - hopes to keep it in DAMPED tomorrow (when not locked)
• bottom left - HPI_HAM6 guardian - in READY, since it's locked
• top righ - HAM6 ISI
• bottom right - HAM6 HEPI

Status:  Discussed smoke from nearby fires.  DetChar F2F starts this morning.  HAM6 vent plan for the week--most likely crack bolts on HAM6 on Tues, BUT we will only start this work when we confirm no dust/smoke issues in the LVEA.

Due to upcoming vent work, we did not go through status of the SEI & ISI.

Bake RGA at end station might start today (Kyle)

HAM6 ISI:  should be isolated or atleast damped (see Hugh)

For Tuesday Maintenance:  Timing work and PI model work were mentioned as primary tasks.  PSL work will cover Mon/Tues per work permit.

For any tours, please remember to fill out a work permit & check with the operator as to whether it is OK (if there is active work in an open chamber we may want to limit tours at that time.)

Visitors This Week:  DetChar people, visitors from China (Fred), KAGRA visitors for PCal (Kiwamu), Krishna/Micahel from UofWA (Hugh)

Attached are trends of the LVEA Dust Monitors.  

• Dust Monitor #6 is in the HAM6 Cleanroom
• Dust Monitor #10 is in the Beer Garden

On the 24hr trend, there was a little bit of dust in the LVEA between 8am-1pm, but in the cleanroom, were have been near zero for the last 24 hrs.

On the 4-day trend, the only dust in the HAM6 cleanroom was from technical cleaning on Friday.

Jeff is going to take some more readings outside.

During the drive in one can certainly see a smoldering Rattlesnake Mountain (did not see much activity [i.e. active fires/personnel] by eye).  From our End-Y building looks like the fire was about 6-7 miles away.

Fred mentioned there were no issues mentioned on the radio with regards to the fire and the Hanford Site.  Ryan pointed me to the hanford.gov advisory site (there is a banner link which you can get the following info about yesterday's nearby fire):

Terra, Nutsinee

Terra, Evan, Stefan

With the spot move script from last alog we could reliably maximize the recycling gain. We reached 30 (as measured by LSC-PR_GAIN - Evan claims this is more like 35).

The instability in AS90 could be removed by manually aligning the SRM - but POP18 got worse when doing so.

Interestingly - the same move that fixes the recycling gain at 50W, has almost no impact on the recycling gain at 2W.

For reference, the last lock loss was at about 6:15 UTC on Aug 1st.

We had to adjust the soft loop gains to

            ezca['ASC-DSOFT_P_GAIN'] = 0.5 # 3 killed it
            ezca['ASC-DSOFT_Y_GAIN'] = 3  # oscillates at lower UGF when much lower
            ezca['ASC-CSOFT_P_GAIN'] = 0.5 # designed differently
            ezca['ASC-CSOFT_Y_GAIN'] = 3  # oscillates at lower UGF when much lower
 

We reached 50W relatively stable and reliable, and were able to use the POP A offsets to drag the interferometer around.

I recorded that particular alignment vector, and stuck it into a script. This now allows moving the full interferometer in this degree of freedom very rapidly, without taxing the soft loops too much.

We see a clear recycling gain dependence on this degree of freedom. However, one oddity is that when we reach the maximum recycling gain, the AS90 buildup becomes very ratty. I am wondering whether we get close to clipping somewhere with this move, or whether it is related to our SRM alignment dither.

The alignment vector for this move is

# pit
gppr3=0.07
gpprm=-13.2
gpim4=-325
gpsr2=-2.39
gpsrm=3.2
gpex=-0.7
gpey=-0.7
gpix=-0.45
gpiy=-0.35

# yaw
gypr3=0.028
gyprm=11.07
gyim4=61.22
gysr2=2.22
gysrm=2.74
gyex=-0.5
gyey=+0.5
gyix=+0.32
gyiy=-0.31
 

Sheila, Stefan

Based on yesterday's measurement, the ASC PUM to TOP x-over was at 0.002Hz and 0.0029Hz (before the 10dB increase in the script).

I added gains of 50 for pitch and 34.5 (for now in the L1_LOCK modules) to move this x-over to 0.1Hz.

The only loops for which this 0.1Hz offload is critical are the soft loops. Thuis I added z=0.01Hz:p=0.1Hz lead filters to all soft loops. I also turned the soft loop bandwidth up by 3x.

The guardian now sets those gains before turning on the soft loops - and sets them back in the down state.

We still should do some more loop tuning for the soft loops - their response should be sped up

We got to 50W - but lost it when moving the PRM - the soft loops seem to be still too slow to follow, resulting in the Izumi-Ballmer instablility.

Stefan, Sheila

We had a lockloss last night during the INCREASE_POWER state when the ISS third loop was coming on.  This is done by a function that had a ten second sleep in a while loop, so the guardian didn't recognize the lockloss for about 90 seconds, in which time ETMY and ITMX suspesnions tripped.  Stage 2 of both ISIs tripped after the suspensions had been tripped for about an hour.  Screenshot shows the sequence of events, first a 90 minute time span then in a 2 minute time span around the lockloss. 

To make sure that in the future the guardian will recognize the guardian more promptly, we moved the ISS third loop engagement to the IMC_LOCK guardian which currently handles the ISS second loop engagement.  The ISC_LOCK guardian just requests this now in the increase power state when it reaches 10 Watts.  

We also looked at why the lock broke.  One potential explanation is that there are large glitches in both DHARD P and Y when the boosts get engaged about 3 seoncds before lockloss (screenshot attached).  These are simple single pole single zero boosts, which were Always on with a 3 second ramp for engaging.  We changed the ramps to 0.2 seconds to see if by ramping faster than any features in the filter we can avoid glitches.  The real cause of the lockloss might have been changes to the offloading that weren't compatible with the guardian as it was last night, but in any case we would like to avoid these large glitches when engaging the boosts.  

I started to look into a faster ASC UIM to TOP stage offloading.

The DC gain ratio of the DRIVEALIGN filter outputs is about PUM:TOP = 23:20000 (pitch) and 30:20000 (yaw). There is a slight mismatch between the optics, but that seems to be taken care of with the CAL filter (FM1 in M0_LOCK).

With that gain ratio, and with FM1, FM4 and FM5 on in the MO_LOCK banks, the current x-over between TOP and PUM is about 0.002Hz (pitch) and 0.0029Hz (yaw).

This is very low. I verified that with 10x the gain in all 4 soft loops (gain=0.45 instead of 0.045) and 10x the offloading gain to top, the soop[s still seem stable. I believe the reason we are not running with that during the angaging these loops tax the SRC loops.

I added 0Hz:0.1Hz AC couplers into the L2 DRIVEALIGN P2P and Y2Y banks, as well as 0.002Hz:0.1Hz lead filters into Mo DRIVEALIGN P2P  banks ( 0.0029Hz:0.1Hz lead filters for Y2Y). These filters were successfully engaged in full lock. Somewhere however the guardian turns up the offloading gain by 10dB, making these filters unstable... So for now they are not in guardian.

Unfortunately, the pwer-up is still failing randomly. Not sure whether that's the 40mph wind, the numerous locked-down hepi's or the increased noise on RF DARM.

Temps look good -> made minor variac adjustments to midstation-destined RGA components that are currently "grafted" onto VBOB (exploiting available clean pump system to bake this sub-assembly and to get a preliminary look at the suspected "dirty" Nitrogen and Krypton calibration gases following bake cycle)

Jenne, Stefan

- Adjusted the PRC1 input matrix form REFL to match the input matrix from POP X at 2Watt. THuardian now has a switch : lscparams.use_popx_wfs (1= use POP X, 0= use REFL WFS). Left it on POP X for now.

- Adjusted the dither input matrix for the SRM to maximize the ratio AS90/POP90. This was done with the gains -0.013 for AS90 and 0.5 for POP 90. (The gain ratio is simply the DC ratio of POP90 and AS90: 17:650.)

- We next tried to establich a stable locking point for the PRM alignment - but unfortunately repeatedly lost in during power-up.

I've had a nagging in the back of my mind for a little bit now, so I looked at the input steering mirror positions from a good time during O1 as compared to now.  This doesn't account for everything yet, since I haven't looked at the pointing of the IMC, but the current OSEM readbacks of the IMs are significantly different than they were during O1.  I wonder if part of our whole problem has been that we just have different enough input pointing that we're having PRC gain issues.

Anyhow, I'll meditate more and look at IMC pointing, but then maybe I'll try moving the IMs back to these Sept 24-29 positions on Sunday.

As a proxy for PRCgain since we didn't have the handy channel that we do now, I plot the TR_X_NORM.  On the left side of the screenshot are the September values, including the TR_X_NORM at about 1320 counts.  When we are at 50W and our (new) PRC gain channel reads 25, TR_X_NORM is at about 900 counts.

Evan G., Jeff K., Evan H.

Summary:
We have updated the L2-L3 crossover filters to reduce the effect of the L2 actuation on the superactuator at frequencies above the L2-L3 crossover. In addition we have turned off the vStopB filter in the L2 drivealign bank. The old filters remain in the filter bank, but are turned off in case we need to revert. The SDF has been updated to reflect these changes.

Details:
The old lowpass [ :30], and highpass [0:30] filters were allowing too much interaction of the L2 stage on the frequencies above the crossover. We needed more aggresive rolloff of the L2 stage. A new design was used to develop prototype lowpass and highpass filters that were then normalized to produce complementary filters. The trick was to add some additional filtering in addition to the single pole or single-zero/pole design.

The Matlab zpk prototype lowpass is:

    29.804 (s^2 + 792s + 4.077e06)
  ----------------------------------
  (s+94.25) (s^2 + 1192s + 1.304e06)

and the prototype highpass is:

  0.68377 s (s+0.4816) (s+40.81)
  -------------------------------
  (s+125.7) (s^2 + 35.75s + 1174)

The resulting complementary filters are the lowpass:

      43.587 (s+125.7) (s^2 + 35.75s + 1174) (s^2 + 792s + 4.077e06)
  ----------------------------------------------------------------------
  (s^2 + 27.96s + 1124) (s^2 + 256s + 1.986e04) (s^2 + 1087s + 1.175e06)

and the complementary highpass:

        s (s+94.25) (s+40.81) (s+0.4816) (s^2 + 1192s + 1.304e06)
  ----------------------------------------------------------------------
  (s^2 + 27.96s + 1124) (s^2 + 256s + 1.986e04) (s^2 + 1087s + 1.175e06)

Putting this into Foton yields the lowpass design:

zpk([63.027+i*315.14;63.027-i*315.14;2.8446+i*4.6522;2.8446-i*4.6522;20], [20.375+i*9.3751;20.375-i*9.3751;2.2252+i*4.8498;2.2252-i*4.8498;86.475+i*149.24; 86.475-i*149.24],1,"n")

and the highpass design (properly normalized to 1 at high frequency):

zpk([94.821+i*155.07;94.821-i*155.07;0;0.076645;6.4948;15], [20.375+i*9.3751;20.375-i*9.3751;2.2252+i*4.8498;2.2252-i*4.8498;86.475+i*149.24; 86.475-i*149.24],1,"n")gain(0.000578996)

These are installed in the L2 and L3 drivealign banks as L2L3LP and L2L3HP, respectively.

We have compared the old and new design and the effect on the DARM actuation and DARM open loop gain and closed loop suppression, and we are satisfied with the improved filter design. Attached is a series of figures to illustrate the new design (see the plots.pdf file)

Figure 1: actuation plant transfer function (left plots) and the PUM/TST ratio (right plots)
Figure 2: current state of the distribution filters (before any modification)
Figure 3: current state of the actuation authority (before any modification) -- note the black curve has ripples in the 100-900 Hz band
Figure 4: new crossover filter design -- prototype and final design
Figure 5: new distribution filter design from Matlab continuous filters -- note that we turn off the vStopB filter module in the L2 drivealign bank
Figure 6: new actuation authority from the Matlab filters -- note the improvement in the black curve with much less ripple than before
Figure 7: using new design, L2 and L3 actuation transfer functions: drive at input to L3 lock bank and look at L3 response for both L2 and L3 paths and taking the sum
Figure 8: using new design, the ratio of the two single curves in figure 7 -- note the transfer function is unity at 23.3 Hz and with a phase margin of 67 degrees
Figure 9: new distribution filter design as implemented in Foton -- note that these are the same as the Matlab design
Figure 10: new actuation authority from the Foton filters -- again, same as before
Figure 11: compare the DARM open loop gain and closed loop suppression from the current state (old) and using the new filter design and also when there is no SRC detuning -- note there was a lingering 400 Hz notch filter in the ER9 run that shouldn't have been there (Evan H to make sure this is off) and this model takes into account the increase of the digital DARM gain to 1400. The blue curves have little ripple above 100 Hz, and the phase margin is actually a little larger than before, about 48 degrees. The gain peaking is roughly the same as before ~4 dB.
Figure 12: A final acutation authority plot showing the new total is much improved over the old total, especially in the range of 100-900 Hz (see zoom on right hand plots).

For the record, also attached are screen shots of the L2/L3 drivealign filter banks before and after the changes as well as a screenshot of the SDF.

State of H1: earthquake, unlocked, ISC_LOCK set to Down, IMC_LOCK set to Offline

Activities:

• HAM6 vent prep
• EY ion pump power supply issues - update: power supply is off for the weekend
• EY leak testing

Since EQ:

• 21:50UTC - Gerardo to EX - and back at 22:35UTC
• 22:00UTC - Marc in LVEA - at HAM5/6 terminating cables - out as of 4:10PM PT
• 22:40UTC - Jenne/David in LVEA - looking for Marc - out as of 4:10PM PT
• 22:42UTC - RichA/guest in LVEA - prep for filming in the LVEA tomorrow - still in LVEA

The OMC alignment servos keep turning on, even though the OMC guardian is paused.  I found that it gets turned on in engage_wfs_centering whenever we do any ASC since it's in a generate states function.  So, I have commented out that line, but we must uncomment it when we have the OMC back and are ready to use it again.  It is currently line 113 in ISC_GEN_STATES.