david.barker@LIGO.ORG - posted 13:57, Wednesday 28 August 2024 (79774)
Restarted alert system, removed Austin's entry.
Closes FAMIS 26005 , last checked in alog79586
HAMs:
Things look much calmer now, except for HAMs 7 & 8 still look as they did previously.
BSCs:
Things again are much calmer.
We have a preliminary report for the calibration status of LHO after the vent recovery, based on the measurement taken in LHO:79691. The report can be found here. Taking a look at the PCALY to (GDS-CALIB_STRAIN_CLEAN) DARM broadband injection, there is an elevated amount of error between 30 and 60Hz. It would be worth considering to update the calibration at the next available opportunity, but it isn't pushing us beyond 10% mag/10deg phase yet. This is also corroborated by the monitoring page here. N.B. I used pcal to GDS instead of to DELTAL_EXTERNAL because the GDS channel is already calibrated to meters for us.
Since the ZM4 DAC output was close to saturation, I offloaded ZM4 yaw with SQZ ASC ON. The SQZ ASC seems to work and keeps good squeezing level. I reduced ZM4 yaw from -1408 to -1108 and the SQZ ASC moved ZM5 and ZM6. Then I offloaded AS42 ASC. I could reduce ZM4 yaw more, but lockloss happened. I accepted the attached SDF.
We are now running an initial alignment and will start main locking shortly.
We took some no SQZ time from 1600-1610 UTC today. Attached are comparisons of that time against 1455UTC today when we had 120Mpc range, and from yesterday with a better 140Mpc.
Latest output of the 'o4' script, TJ requested I add the GPS times.
Yesterday evening during some high wind time, Ryan C added some code to ISC_LOCK that he has been testing out in our TEST guardian node (mentioned in his summary alog79749). While successful in that node, ISC_LOCK started having connection issues for channels that exsist in the SDF. Stopping and starting the node resulted in different channels that it couldn't connect to each time. I've removed this from ISC_LOCK for now, but kept the main function in ISC_library.py. Ryan and I will try to figure out some different testing to be done before implementing again.
The well pump is currently running to replenish the fire water tank as we are bordering on the low-limit threshold. Coordination with the operator did take place ahead of the start command. Pump will run for four hours beginning at 9:28 pst. C. Soike T. Guidry
Wed Aug 28 08:10:00 2024 INFO: Fill completed in 9min 56secs
Richard confirmed a good fill via camera.
TITLE: 08/28 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Observing at 137Mpc
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 5mph Gusts, 4mph 5min avg
Primary useism: 0.01 μm/s
Secondary useism: 0.11 μm/s
QUICK SUMMARY: Locked for 8 hours, calm environment so far. Verbal has record of PI31 ringing up 30min ago, but it must have damped. I wonder if this gain also needs to be changed though. Our range is quite erradic for this lock. It sounds like this is due to the SQZ whistles that Sheila mentioned in her alog79757. Planned commissioning today from 9-12 PST.
Looking at the FC WFS plot here: summary pages SQZ subsystems you can see that the squeezer whistles have reappeared. This is a possible explanation for the nonstationarity and poor range of DARM overnight (There's an issue with the summary page of last night's range right now, but it should appear here: range summary. However, looking at the previous days, the squeezer whistles appeared on the 25th and have been getting more frequent, but we didn't see poor range until last night.
As Ryan Crouch noted, last night was windy, but the wind died down around the time the IFO lost lock, and the last 6 hours have been calm with poor range.
It looks like it was for a picomotor (from SYS); screenshot attached.
TITLE: 08/28 Eve Shift: 2300-0500 UTC (1600-2200 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Corey
SHIFT SUMMARY: High winds made locking very difficult, they're dying down as the shift is ending and I just finished an IA so I'm hopeful the IFO will be able to lock itself as we just got DRMI offloaded at 05:00UTC
Lock #1
Lock #2
Lock#3:
I was able to new SRCL and MICH feedforward using the iterative method. Measurements were taken in alog 79693.
The current MICH FF is performing well except for below 30 Hz, so that's where I targeted in my fit. SRCL needed some improvement everywhere.
The new MICH filter is saved and loaded in FM8 and the new SRCL filter is saved and loaded in FM5. They are both labeled with the date '8-27-24'.
To compare how these filters are performing, I would use the templates Oli saved in the above alog, but first save the live trace as a reference to compare against.
I have not yet done the PRCL fit because I want to see how SRCL performs, and maybe redo the PRCL injections before trying the fit.
Here is some more information about the fits.
First, my MICH fit caused a lockloss this morning because I forgot to the check the phase on the filter. Sometimes, the phase gets flipped during the fitting. Usually, I compare the sign of the phase with the previous filter in foton and adjust accordingly, but I forgot to do it this time (rookie mistake). I have double checked the new SRCL filter phase and fixed the MICH phase.
Attached are two screenshots of the fittings. First, the MICH fitting compares the current filter, labeled as "reference 1" with the red trace which represents the new fit. The bottom right plot compares the fit residuals. You can see from this plot that the most improvement occurs at low frequency, with some small improvement at mid frequency. The SRCL fitting has many more traces, but compare the orange "current fit" trace with the trace labeled "best with less HF gain". This has more improvement almost everywhere. There is an increase in gain at high frequency, but it is less than an order of magnitude, so I think it's ok. The new fit also has reduced the high Q feature around 300 Hz that was potentially injecting noise. There is a factor 5-10 improvement between 10-50 Hz that will help the most.
These have now been tested, SRCL passes, MICH fails.
The SRCL screenshot compares Oli's SRCL measurement from four days ago with the "current" filter, and the new "trial" filter that I applied today. There is clear improvement everywhere (except for a small worsening between 100-200 Hz), so I think we should use this filter.
The MICH screenshot compares Oli's MICH measurement from four days ago with the "current" filter, along with "no FF" and "trial". The trial did what I promised, and reduced the coupling between 10-20 Hz, but clearly at the sacrifice of the noise everywhere else. I think we should stay with "current".
I am changing the guardian to select this new SRCL filter (FM5) in "lownoise length control" and the gain back to 1. There will be an SDF observing diff for SRCLFF1 that can be accepted by the operator.
Some thoughts about MICH FF:
In my opinion the hardest region to fit is between 10-20 Hz because of the presence of some high Q features, such as around 17 Hz. It would be worth considering what is causing those features- perhaps some bounce/roll notches. Do we still need those notches, or could they be briefly turned off during a feedforward injection? That might make the low frequency portion easier to fit and therefore easier to achieve good subtraction 10-30 Hz.
Looks like these are maybe BS M2 LOCK L FM10. We can try turning them off I suspect. Foton says they have a 2 second ramp, so should be okay to turn off just before the measurement (I'm not sure if we need them all the time, but maybe we do).
Today Sheila took another injection of PRCL for me so I could fit a new feedforward. The fit looked promising, however once it engaged it apparently caused oscillations everywhere, and I turned it off fast enough to avoid lockloss (thanks Corey and Ibrahim!). I checked the phase and gains beforehand, no high Q features, etc so I don't know what could be the issue.
In the last lock, PI31 at 10.428 kHz rang up as shown in the first attachment. Although it was damped by SUS_PI guardian, PI31 RMSMON reached close to 2000. The frequency of this PI is 10427.67 Hz as shown in the second attachment. This is within the input bandpass filter (425-431 Hz after downconversion by 10kHz). I increased damping gain for PI31 from 1000 to 10000 in SUS_PI guardian.
EDIT: This gain increase was too much. I reverted the damping gain to 1000 in SUS_PI guardian.
WP12061 Upgrade RCG h1susex, remove LIGO-DAC delays
EJ, Erik, Jonathan, Dave, Daniel, Marc:
h1susex RCG was upgraded to a custom 5.3.0 specifically for the IOP to remove a delay in the new LIGO 28AO32 DAC. We compliled all of the user models with this RCG as well.
Our first restart was a simple model restart, but we got Dolphin errors. So our second restart was to fence h1susex from the Dolphin fabric and power cycle h1susex via IPMI. After this the models started with no issues.
No DAQ restart was required.
Code path is /opt/rtcds/rtscore/advLigoRTS-5.3.0_dacfix
WP12063 Alert System
Dave:
The locklossalert system code was modified to permit an alert window which spans over midnight, needed for the new owl shift hours.
Note that the business/every day filter is applied after the minute-in-day filter, so a window starting Friday evening and extending into Saturday morning will cut off at midnight if business days only is selected.
One other change:
For everyone subscribed to Guardian alerts, if for any reason the Guardian EPICS record cannot be read (node down, h1guardian1 down) the alert will now default to SEND.
Guardian Reboot
TJ:
h1guardian1 was rebooted at 08:21 PDT. All nodes except TEST came back automatically. TJ worked on TEST and got it going.
MSR Rack Cleanup
Jonathan:
Jonathan removed two test switches from the MSR racks which are no longer needed.
I updated the o4 script which reports where we are in O4 and reminds us of important dates (break start/end, vent start/end).
Tue27Aug2024
LOC TIME HOSTNAME MODEL/REBOOT
08:50:45 h1susex h1iopsusex <<< 1st try, restart model
08:50:59 h1susex h1susetmx
08:51:13 h1susex h1sustmsx
08:51:27 h1susex h1susetmxpi
08:57:11 h1susex h1iopsusex <<< 2nd try, reboot
08:57:24 h1susex h1susetmx
08:57:37 h1susex h1sustmsx
08:57:50 h1susex h1susetmxpi
