Jim W, TJ S
The SEI_ENV node will now automatically handle USEISM and USEISM_EARTHQUAKE state transitions based on environment conditions. Users will now request the AUTOMATIC state, and the node should determine where to go from there. This AUTOMATIC state will only be the request, it should never stay in this state for any length of time. Normal earthquake transitions will occur, and maintenance mode is also unaffected.
Jim and I have discussed for some time about how best to handle the useism vs wind conditions. We're going back to a simple method of assuming that we are in a windy/calm state unless the max of H1:ISI-GND_STS_ITMY_Z_BLRMS_100M_300M.mean,m-trend over the last two hours is above 800. We will also subtract out any times where the peakmon channel is above 1000 to avoid any bleed into this band. It's possible that we might look at the slopes of some type of linear regression fit and compare that over some length of time and then determine if it should be switching or not, but we'll start small and get there later.
Camilla, Erik, TJ, Dave:
My original HWS camera control code queried the camera state every minute (camera running or stopped waiting for an external trigger). We found that the HWS servers seemed to be unstable when this code was running over the holiday break. We suspected a possible race condition between the camera control code and the HWS frame grabber. To test this, we stopped the camera control code late last week, and have had no server issues since.
Today, on Camilla's request, I have modified the code to only query the camera status at code startup, and after the camera has been turned on/off (as verification). This means the new code will only send commands to the camera when H1 changes its lock state.
[Measurements attached]
FAMIS LINK: 25973
BSC CPS: Received following from CPS script terminal output:
NOTE: If one was to follow the above channel as an example, it would seem several other BSCs had CPS' with spectra similarly high by eye:
HAM CPS: Looks good.
Maintenance activities have wrapped up, but the wind is still gusting in the 40s mph and sometimes above 60mph, and the useism continues to grow. We will hold off locking until the environment calms down a bit.
Mid X alarmed this morning due to low temperature in the main room. The heating coil for this space was limited to a 95 degree discharge air temperature in the tuning parameters. This meant even with a 100% heating call in the space, the coil was only allowed to run ~50% to prevent it from putting out over 95 F. I increased the discharge air temperature limit from 95 F to 115 F to allow the heater to reach 100% capacity. Space has returned to setpoint.
[Dave, Erik]
CDS Login was updated to fix EPICS problems from a previous update which crashed the EDC. See here for details on the crash: https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=74959
Lockloss alerts were temporarily down during the upgrade.
Workaround used to keep things running in the interim wree removed.
On 26th Dec 75041, IFO unlocked during ETMX stage of in-lock charge measurements in 75171 I adjusted guardian code to make the transition more robust.
The ITMX, ITMY, ETMY measurements did run, analysis attached. ETMX hasn't ran since November. We will try to run them all this week in commissioning time.
Note that the x-axis of the plots isn't a to-scale time axis, we will try and change this.
Tue Jan 09 10:10:24 2024 INFO: Fill completed in 10min 20secs
Gerardo confirmed a good fill curbside.
We updated the tzdata and leapseconds database on some frozen infrastructure systems, including the FE computers, DAQD, and guardian machines. This was done by an apt-get install tzdata, not a general churn in the software versions.
Vicky noticed that H1:SQZ-SHG_FIBR_REJECTED_DC_POWERMON is high, this is the rejected polarization polarization sent though the OPO pump fiber and measured in HAM7, trend over time attached.
Followed instructions in 71761 to reduce it from 0.55 to 0.03, plot. With ISS on, H1:SQZ-OPO_ISS_CONTROLMON is at 4.1V
With SQZ_OPO_LR in DOWN, H1:SQZ-OPO_REFL_DC_POWER increased from 2.2 to 2.3. Our maximum after on table AOM and fiber alignment is around 2.9V (last done August, September and November 74479).
Lockloss at 08:35UTC from an earthquake, high winds as well at the time.
TITLE: 01/09 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Wind
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
SEI_ENV state: MAINTENANCE
Wind: 33mph Gusts, 21mph 5min avg
Primary useism: 0.14 μm/s
Secondary useism: 0.83 μm/s
QUICK SUMMARY: Lost lock 7.5 hours ago and Ryan was unable to relock with the high wind and high useism. Maintenance activities have started.
Workstations were updated and rebooted. This was an OS package update only. Conda packages were not updated.
H1 called for assistance at 10:15UTC, it seems to be stuck in IA as we're still ringing down from an earthquake and winds have picked up which is making locking Yarm a struggle.
IA finished at 11:12UTC, back to locking. I had to help yarm lock during IA.
With wind increasing with gusts up to 40mph I've taken SEI back to WINDY from MICROSEISM
Wind gusts up to 60mph! ALS is having lots of difficulty locking due to the wind and high microseism. The forecast on windy.com says the wind is likely stay at the same level for the rest of the morning and increase further, peaking around 11am today. So I'm not super hopeful of the IFO getting relocked, unless the wind dies down for a short period.
As in 74872 and 74741, I have taken PEM_MAG_INJ and SUS_CHARGE from WAITING to DOWN so that they do not run tomorrow. Instead, tomorrow Louis and Sheila will try the risky DARM loop swaps and calibration from 7amPT. To re-enable the automated measurements, the nodes should be requested to INJECTIONS_COMPLETE before next Tuesday.
IFO was unlocked from wind this morning. Re-requested both guardians to INJECTIONS_COMPLETE.
State of H1: Observing at 159Mpc
Very quiet shift with H1 observing the entire time except at 03:25 UTC when SQZ unlocked (SQZ_OPO_LR Guardian reported "PZT voltage limits exceeded"). Guardians were able to bring everything back automatically and observing was resumed within 2 minutes. BNS range improved by 5-6Mpc after this event.
We have a checker in SQZ_MAMNGER to relock the OPO if the PZT is not in 50-110V range when IFO is down to prevent this. The PZt changed too fast for this checker to help though, probably caused by a 0.4degF LVEA temperature change in zone 4 at the time.
The range increase appears to be because the SQZ angle was in a bad place before relock (220 degrees rather than nominal 180), see attached. Unsure if we expect the OPO PZT changing to effect SQZ angle.
This may be improved by moving the ADF closer to where we want to optimize (200Hz?). Currently the ADF at 1.3kHz but best range is with SQZ not optimized at 1.3kHz 75151. There could be two zero ADF servo crossings at 1.3kHz, one with good 300Hz SQZ and one with bad 300Hz SQZ and sometimes the servo takes us to the wrong one.
Attached is DARM before and after this relock.