WAP link on the CDS O3 Overview now opens the Unifi MEDM. Remember that on the detailed screen RED=OFF and on the O3 Overview screen the opposite is the case, RED=ON (non nominal for observation).
Back to Observing at 2344UTC, after some FSS corrective maintenance.
Locked for 9 hours. We had a small commissioning break, but are back to Observing.
[Sheila, Jenne]
We took a few tens of minutes this morning to check how far and fast we can move the SRC spot by changing the AS_C offset. Once we had those numbers, we started a script to slowly move the offsets while we're Observing. The script started at 18:21 UTC.
This measurement is now over. We lost lock during the second to last move that this script makes (going from -0.6 P 0.6 Y offsets to -0.3 P 0.3 Y). Perhaps those last moves were too large to take at once.
The pdf attachment shows the median of the kappa c value reported by the front end in the last 60 seconds of each measurement time. (Jenne's script moved the offset, waited 300 seconds, I'm grabing the last 60 seconds of the 300 seconds.). There is about a 0.5% change in kappa_c over the scan. When we are servoing the power on the DCPDs to 20mA and changning the amount of junk light photocurrent, the optical gain would be proportional to sqrt(20mA -junk light photocurrent). If we use 1.3mA of junk light estimated in 55287 then we are reducing the junk light from 1.3 to 1.1mA, or by about 15%.
While there is some noise in the data, the optical gain does seem higher for offsets in the negative pitch direction (as the beam moves up on AS_C). Our nominal position is pit -0.16 and yaw -0.06.
The second attached png is a plot of the time that we changed the modulation depth and changed the darm offset in 55405. In this offset test we are moving AS_C, SR2 and SRM much further for a smaller change in the apparent junk light compared to the modulation depth change test.
Looks like the INJ_TRANS node had a failed injection last night, and it has been stuck in the FAILED_DURING_ACTIVE_INJECT state since. I don't see anywhere in the code a way for it to get out of this state without going to either manual, or INJECT_KILL first. So I brought it to INJECT_KILL and then back to INJECT_SUCCESS.
Log for INJ_TRANS during failed injection is attached.
Out at 1734 UTC
Back to Observing at 1755 UTC.
Bubba G, Chris S, Tyler G A focal point of yesterdays extended maintenance phase was exercising the footing of the HAM 7 chamber. This was performed partially to ensure that future efforts to hoist and move the HAM 7 chamber could proceed without issues related to any adherances between the footing and the grout below. Per Bubbas request, I fabricated two crossmembers that span the framework of the HAM chamber. These crossmembers utilize existing holes within the footing and allowed for the use of two single-stage 10 ton Enerpac hydraulic rams to be placed at opposing ends of the chamber for lifting. With all 24 anchor bolts loosened and raised approximately .5"- 1". Pressure was applied to the jacks and subsequently the crossmembers. With minimal effort, the chamber was successfully lifted and freed from its grout footing. As requested by Chandra, All 4 of the seismic bellows were adequately covered with ameristat and taped to ensure that no disturbed particulate might contaminate these areas during the work. Additionally, the near side bellows of HAM 8 were also sealed off.
You guys make it look so easy! Tagging VE.
TITLE: 03/04 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 115Mpc
OUTGOING OPERATOR: Jim
CURRENT ENVIRONMENT:
SEI_CONF state: WINDY
Wind: 2mph Gusts, 1mph 5min avg
Primary useism: 0.04 μm/s
Secondary useism: 0.39 μm/s
QUICK SUMMARY: Locked for 5 hours, winds have calmed down on site.
Just got back to observing, but getting locked tonight has been difficult. Each lockloss causes the FSS to start oscillating, I couldn't even get it to calm down when Corey left, so I had to call Keita for some help.
So far the process has been: disable ISS autolocker, toggle FSS autolocker, increase FSS oscillation threshold (I put it at 4), wait for FSS to settle down, re-engage ISS autolocker and cross fingers. If that worked, the oscillation threshold was reverted Usually by that point the arms had locked and we were back on our way, but each lockloss was a repeat of this process. I also think Keita must have paused the FSS guardian, but I can't remember if he told me that he did that.
Hopefully, the winds don't get any worse, I don't want to be doing this all night. Seems okay for now, I haven't had to touch anything else.
Jim called, he couldn't lock FSS for a very long time.
When I logged in, I saw that the FSS auto locker was trapped by a sideband resonance (I assume). It would increase the temperature, hit the sideband resonance, which kicked the servo even though the transmission threshold was not met (because the analog servo is always on when the autolocker is on), temperature goes down (I don't understand this part fully), and the auto locker still scans the temperature up, same thing happens again, and after 5 to 10 repetitions the scan direction flips and the temperature goes down, i.e. the scan never hit the carrier though the setting for the scan should have allowed the autolocker to hit the carrier.
What I did was to
TITLE: 03/04 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Aligning
INCOMING OPERATOR: Jim
SHIFT SUMMARY:
Rough recovery after the Maintenance Day with only ~1hr of observing and H1 still down due to the PSL's FSS.
LOG:
H1 lost lock with no noticeable environmental issues. Haven't had time to see if FSS could be the culprit since it has been causing grief as of late.
It's been almost 30min and I have not been able to lock the FSS with Jason's instructions from January. Keita has issues with the FSS this morning and tried different ways of bringing it back and mentioned someone might need to go out into the PSL Room.
H1 can not come back without the FSS, so this will be down time due to the FSS.
(Driggers, Gray, Kawabe)
Whew! What a day! Started off the shift with Maintenance still going, but then moved immediately into running an alignment.
The alignment was OK, but the FSS started oscillating during it. It took a good 20min to get it back. After the alignment, made it to DRMI and the flashes/alignment looked good, but basically for the next 4hrs, could not get it to lock (See Jenne's alog).
The FSS has been causing downtime during this shift for any time the IMC breaks lock. Sometimes it comes back within a few minutes, but there have been other times when it's taken over 15min. (maybe it's due to the SDF changes listed below?)
Eventually got H1 to Observing at 6:34!
There were SDF diffs (ATTACHED) for:
GRD_IFO was in the Automatic Operation. So as soon as the SDFs were cleared, Intent Bit went to OBSERVE on its own. (I switched it back to MANAGED)
As maintenance was closing out, but the Vac team was still working, I wanted to try some SEI -> PRCL measurements for a potential redo of Ryan and I's 2010 feedforward. But, since the gate valves were closed, I changed the lscparams gaveValves flag (on ~line 99) to True, indicating that a gate valve was closed. I forgot to change it back, and that seems to have caused the several hours of grief that TJ, Corey, and Keita worked on troubleshooting. A big symptom of this grief was that PRMI would lock, but DRMI just never would. Once I realized / remembered after being on the phone with the control room for a while, Corey and I changed it back to it's usual value of False, and DRMI immediately caught lock.
As part of troubleshooting, I had Corey do a very careful initial alignment (even though he had already done one, and thought the DRMI flashes looked fine).
It seems like the acquisition sequence is going forward nicely now. Sorry :/
Once activities ended for Maintenance Day, immediately went for an Initial Alignment.
One item to note for alignment, was there was a lockloss where the PSL's FSS went into an oscillating state. Toggled the FSS Autolocker. And then Turned Off the autolocker for ISS 1st Loop (per instructions in January).
Currently have made a few attempts at locking, and have not been able to lock DRMI (other than a short 2-sec one) so far (even with nice flashing). Have had 1-2 more instances of the FSS going into oscillation and this need operator attention. Going to give DRMI locking a few more attempts with PRMI locks as well. After that, may try another alignment?
Definitely not rosy after Maintenance Day and something may be amiss. Will post update later.
Additional NOTE:
CHECK MICH FRINGES appears to not work in that it looks like it's trying to lock a bright fringe (or just having trouble locking on a dark fringe). Either way, it is not clear how to adjust the BS when ISC LOCK is in the CHECK MICH FRINGES state. (think this has been the case for 2-3 months?).
CORRECTION: Looks like nowadays CHECK_MICH_FRINGES is an automated step (i.e. raises PSL power, dithers BS, then runs Mich Offloaded). So, looks like there is no "manual CHECK_MICH_FRINGES" like what operators used to do a few months ago.
More Notes:
BRSx continues to be in the DAMPING state post-EX noisy work in the VEA. See attached "BRS Health" plots.
Addendum: 3:23 EX signals return to normal & it returns to READY state (& SEI_CONF node's NOMINAL box turns from orange to all green).
On current lock, have had (4) attempts at DRMI with NO locks at all. PRMI locks with no problem.
Contemplating an alignment and then if that doesn't work again, will start calling for help.
Have been noticing in ISC_LOCK log that it says "Unstalling TCS_ITMY_CO2_PWR".
Going to the TCS_ITMY_CO2_PWR node shows it in a loop between "HOLD_POWER" & "ADJUSTING_POWER". Not sure if this is normal, or not, but wanted to mention this oddity (because TCSx does not have this issue).
But this might be a normal/usual thing. Just looking for anything unusual.
