TITLE: 08/30 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 149Mpc
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY:
Lock#1:
Lockloss at 16:18UTC
The squeezer kept losing lock all morning, and the ISS saturated.
Lock#2:
After some investigation while we were relocking it was found that FC1 was having troubles, Fil power cycled the chassis and then swapped the coil driver and satellite box for a spare. But none of these actions were successful to fix the issue.
We required NLN at 17:52, but we stayed out of Observing while the FC1 investigation/work was ongoing. It was narrowed down to an issue with the T3 BOSEM on FC1. An old (2011) ADC card ended up being swapped for a new one (2021) which was the root of issue! Fil swapped back the original coil drivers and satellite box. Alog72558 documenting the process
I took a broadband and simulines calibration measurement during the investigation time, then a second one later on in the afternoon.
Back into Observing at 22:46UTC after squeezer commissioning and MICHFF testing.
LOG:
Start Time | System | Name | Location | Lazer_Haz | Task | Time End |
---|---|---|---|---|---|---|
15:25 | FAC | Cindy | MidX | N | Tech clean | 16:53 |
17:39 | EE | Fil | CER | N | Investigate FC1 | 17:51 |
17:59 | EE | Fil | MidY | N | Get a spare satellite box | 18:22 |
18:22 | EE | Fil | CER | N | Coil driver, Satellite box swap FC1 | 18:50 |
18:50 | FAC | Cindy | Mech room | N | Tech clean | 19:37 |
19:40 | SUS | Fil, Rahul | CER | N | Investigate whether to replace AA chassis | 20:07 |
19:41 | FAC | Cindy | H2 | N | Tech clean | 19:58 |
19:50 | LSC | Elenna | Remote | N | Test new MICHFF, OMC injection | 20:19 |
20:15 | EE | Fil, Dave | MSR, CER | N | HAM7 FC1 AA chassis swap, restore swapped parts (Coil drivers, satellite box) | 20:40 |
20:42 | ASC | Elenna | Remote | N | Measurments, 8Hz line | 21:20 |
20:45 | SQZ | Vicky, Sheila, Naoki | CR | N | SQZ commissioning | 22:28 |
21:16 | FAC | Randy | LVEA, west bay | N | Grab parts | 21:21 |
22:19 | FAC | Tyler | EndY chiller yard | N | Check on chiller | 22:33 |
22:37 | LSC | Elenna, Camilla | Remote | N | Test new filter | 22:45 |
I ran a 2nd calibration sweep today, starting with broadband:
/ligo/groups/cal/H1/measurements/PCALY2DARM_BB/PCALY2DARM_BB_20230830T212846Z.xml
Simulines:
2023-08-30 21:58:00,615 | INFO | Commencing data processing.
2023-08-30 21:58:56,567 | INFO | File written out to: /ligo/groups/cal/H1/measurements/DARMOLG_SS/DARMOLG_SS_20230830T213653Z.hdf5
2023-08-30 21:58:56,585 | INFO | File written out to: /ligo/groups/cal/H1/measurements/PCALY2DARM_SS/PCALY2DARM_SS_20230830T213653Z.hdf5
2023-08-30 21:58:56,611 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L1_SS/SUSETMX_L1_SS_20230830T213653Z.hdf5
2023-08-30 21:58:56,636 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L2_SS/SUSETMX_L2_SS_20230830T213653Z.hdf5
2023-08-30 21:58:56,661 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L3_SS/SUSETMX_L3_SS_20230830T213653Z.hdf5
GPS start: 1377466629.697354
GPS stop: 1377467954.983395
We think this is a more thermalized measure of the IFO after installing the new DARM2 FM8 boost filter, and we'll likely use *this* measurement to inform a calibration update. aLOGs of DARM2 FM8 boost filter change -- LHO:72562 and LHO:72569 Previous unthermalized measurement thta also had the new DARM filter in place -- LHO:72560
This measurement has been processed by pydarm, and can now be found under the report 20230830T213653Z. Attached here for reference. This measurement served as the basis for the update to the calibration on 2023-08-31 -- see LHO:72594. I've measured the OMC DCPD "rough [mA]" to DARM_ERR [ct] transfer function during this measurement, and found the magnitude to be 2475726 [mA/ct] at 5 [Hz]. DTT template is committed to the CalSVN under /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Measurements/FullIFOSensingTFs 2023-08-30_2130UTC_H1_OMCDCPDSUM_to_DARMIN1.xml
Brian and I had a meeting with seismologist at the National Earthquake Information Center in Colorado. They pointed us to a very cool tool called MUSTANG that shows statistics for every seismic station that is fed into the IRIS Data Management Center (DMC).
The tool can be found here:https://services.iris.edu/mustang/
It can be queried by changing the URL at the top, they have a few examples. One interesting rabbit hole we went into was looking at spectrograms for the LHO vault seismometer, known to the IRIS DMC as UW.LIGO. Here are the spectrogram results for the three components:
https://services.iris.edu/mustang/noise-pdf-browser/1/spectrogram?target=UW.LIGO..HH*.M&starttime=2019-03-01&endtime=2023-07-01 (also shown in figure attached)
You can see the seasonality of the microseism and also another periodic low frequency signal of unknown origin. There are tons of interesting things to look at, like the ground motion models:
http://services.iris.edu/mustang/noise-pdf-browser/1/breakout?target=UW.LIGO..HHZ.M&time=2022-01-01T00:00:00&interval=year (also shown in figure attached)
Just putting it out here in case anyone wants to check it out and go down the same rabbit hole Brian and I did.
J. Kissel, L. Dartez Given that we now like Gabriele / Elenna's new DARM2 boost filter in FM8, and it's been turned on permanently as of 2023-08-30 19:03:47 (12:03:47 PDT) -- see LHO:72562 -- we need to update the calibration infrastructure to handle it. Attached is a screenshot of the DARM1 and DARM2 filter banks as we want them now. As such, we've done the following: (1) Copy over the latest H1OMC.txt filter file (where we just have to know that that's where the DARM2 lives) that has the new filter in it over to the calibration SVN's filter archive. $ chans $ cd filter_archive/h1omc/ $ ls -ltr # to find the latest filter file -- is H1OMC_1377459704.txt, and it was modified at 2023-08-30 12:41 PDT $ cp H1OMC_1377459704.txt /ligo/svncommon/CalSVN/aligocalibration/trunk/Common/H1CalFilterArchive/h1omc/ $ cd /ligo/svncommon/CalSVN/aligocalibration/trunk/Common/H1CalFilterArchive/h1omc/ $ svn add H1OMC_1377459704.txt $ svn commit -m "(our commit message)" (2) Update the call to this filter file, and the filter modules used in "the" pydarm parameter file. (a) Make sure the local copy of the pydarm_H1.ini list of "static" parameters is up-to-date with no local changes. $ cd /ligo/groups/cal/H1/ifo/ $ git fetch # use a combination of $ git status $ gitk $ git gui # and just *looking* at the pydarm_H1.ini file to understand where things stand. # We found that there were lots of local uncommitted changes from a major overhaul on July 30 where all comments were stripped from the file and variables were re-arranged # We *think* there was no functional change, so we committed all the changes. $ git add pydarm_H1.ini $ git commit -m "(our commit message)" pydarm_H1.ini # got some git crap about my email address not beign the right email address... had to $ git config --global user.name "Jeff Kissel" $ git config --global user.email jeffrey.kissel@ligo.org $ git commit --amend --reset-author # that popped open another message with vim interface, had to blindly hit enter or :wq, or q a bunch of times... # ran $ git log # to confirm that we've staged the commit appropriately (and Louis assures me I have) $ git push This brings pydarm_H1.ini to git hash 0185c1d7. (b) *Now* make the changes that we actually need to change, $ gedit /ligo/groups/cal/H1/ifo/pydarm_H1.ini # change line 48 digital_filter_modules = 1,2,3,4,7,9,10: 3,4,5,6,7 ==>> digital_filter_modules = 1,2,3,4,7,9,10: 3,4,5,6,7,8 # change line 50 digital_filter_file = cal_data/H1OMC_1336944438.txt ==>> digital_filter_file = cal_data/H1OMC_1377459704.txt $ git add pydarm_H1.ini $ git commit -m "(our commit message)" pydarm_H1.ini $ git push This brings pydarm_H1.ini to git hash 3c57e5f9. (3) We *also* need to make sure that this "cal_data" folder is meaningful. So we have to update a disctionary inside the pydarm_cmd_H1.yaml "pydarm command line infrastructure parameter set," (a) We also checked the local status of this file against the mothership, and we see that there are also lots of uncommitted differences. Louis assures me that he understands these differences, but we can't commit them now. *sigh* (b) So, we changed the local file as it was. # change line 78 digital_filter_file: "Common/H1CalFilterArchive/h1omc/H1OMC_1336944438.txt" ==>> digital_filter_file: "Common/H1CalFilterArchive/h1omc/H1OMC_1377459704.txt" Again, this version of the pydarm_cmd_H1.yaml NOT yet committed.
I ran a calibration sweep this afternoon, starting with the broadband:
/ligo/groups/cal/H1/measurements/PCALY2DARM_BB/PCALY2DARM_BB_20230830T190736Z.xml
Simulines:
2023-08-30 19:36:43,647 | INFO | File written out to: /ligo/groups/cal/H1/measurements/DARMOLG_SS/DARMOLG_SS_20230830T191430Z.hdf5
2023-08-30 19:36:43,663 | INFO | File written out to: /ligo/groups/cal/H1/measurements/PCALY2DARM_SS/PCALY2DARM_SS_20230830T191430Z.hdf5
2023-08-30 19:36:43,674 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L1_SS/SUSETMX_L1_SS_20230830T191430Z.hdf5
2023-08-30 19:36:43,684 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L2_SS/SUSETMX_L2_SS_20230830T191430Z.hdf5
2023-08-30 19:36:43,696 | INFO | File written out to: /ligo/groups/cal/H1/measurements/SUSETMX_L3_SS/SUSETMX_L3_SS_20230830T191430Z.hdf5
GPS start: 1377458086.806734
GPS stop: 1377459422.017018
This is the first calibration measurement with the new DARM boost filter (alog 72565)
... however, we realized too late that the IFO had not yet thermalized during this measurement. As such, we're not confident that this measurement will convey a representative sensing function suitable for an update to the calibration pipeline parameters. So, we will likely throw this *sensing* measurement in the "not yet thermalized" bin. It is still a totally fine actuation measurement. For a more thermalized measurement in the same IFO configuration a few hours later -- see LHO:72573. Attached shows a trend of the TDCFs (specifically the relative optical gain, \kappa_C, and cavity pole frequency, f_CC -- second and third rows) vs. the arm cavity power (bottom row). The first dashed vertical line is the start of this measurement's time. The second dashed vertical line is the start of the LHO:72573 measurement. The arm cavity power is much more like the previous stretch's thermalized arm power at the second vertical dashed line, and less "on the exponential rise." One can see this *less* so in the \kappa_C and f_CC trend, but we feel it better to be safe than sorry and use the LHO:72573 measurement 2.5 hours later.
We've reaquired NLN at 17:52UTC after a lockloss this morning but we're not in observing due to a potential electronics issue on FC1 for the squeeze system which is being investigated.
It seems to be an issue with the T3 BOSEM on FC1
OSEM spectra of T3 BOSEM on FC1 shows that it is very noisy compared to the rest - see pic attached. Fil and I power cycled AA chassis and that did not make any difference. Fil and Dave are currently power cycling HAM7 electronics chain and Dave is also restarting the models (SUSH7).
I have moved all suspensions in HAM7 into SAFE state.
Power cycle of h1sush7 IO Chassis has not fixed the issue, front end computer has been powered off a second time, Fil is heading out to replace the first ADC.
BOSEM T3 is now looking healthy after Fil replaced ADC card, as shown in the spectra attached below. So we are back in action.
Ryan and I have brought all the suspensions in HAM7 to ALIGNED state.
Replacing the first ADC has resolved the FC1 T3 OSEM issue.
Bad ADC (removed) | 110124-03 |
Replacement ADC (installed) | 211109-18 |
Work done under WP 11401.
FRS ticket 29047 filed and closed
https://services1.ligo-la.caltech.edu/FRS/show_bug.cgi?id=29047
Observation time lost due to this fault was approximately 6 hours.
Cosmic Ray analog channels were added as DQ channels to the frame as of noon PST 8/29/2023.
PhotoMultiplier Low 1 = PMLO1_DQ
PhotoMultiplier Low 2 = PMLO2_DQ
PhotoMultiplier High 1 = PMHI1_DQ
PhotoMultiplier High 2 = PMHI2_DQ
Per WP11400
After commissioning on Friday 8/25 we saw a significant jump in range. Initially we were unsure whether it was due to the chiller pump at EY being replaced or due to the commissioning work that Elenna and Gabriele did (72430). Upon further investigation it looks like the range improvement was related to the MICH FF changes and not the chiller pump. NDScope plots attached showing the range related to the FF work, and a plot of DARM just before and after the FF changes.
I forgot to mention that the chiller pump installation finished around 15:30 PDT, +/- 15 minutes. No significant range changes were seen in this timespan.
TITLE: 08/30 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 143Mpc
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 15mph Gusts, 10mph 5min avg
Primary useism: 0.03 μm/s
Secondary useism: 0.07 μm/s
QUICK SUMMARY:
We dropped Observing at 15:16 from the squeezer losing lock, back in Observing at 15:18
Squeezer kept losing lock, eventually it gave the message ISS is off check alog 70050, which I did and followed the instructions and set opo_grTrans_setpoint_uW to 60. I accepted the new OPO_TEC_temp in SDF and brought us back to Observing at 16:03
FC1 seems to have some strange things going on; from screenshot, see the circled FC1_M1_OSEMINF_T3_INMON, which seem to have changed behavior in the past 12 hours, and we see exces smotion in FC1_M1_DAMP_P/R/V, compared to the previous/normal locks.
On FC GREEN trans camera, the beam spot appears visibly dithering in pitch while it is locked. No input suspension ZM1/2/3 or FC1/2 see dithers running, but the DAMP filters on FC1 all seem to have a lot of motion, even with the FC is unlocked.
[Elenna, Dan, Gabriele]
We tested a filter (FM8 in DARM2) that increased the DARM gain below 3-4 Hz, where most of the RMS is accumulated.
The DARM RMS is reduced by a factor of 3. There is no immediate evident effect on the DARM noise aboev 10 Hz. However, it would be useful to test this new filter for a longer time in the future.
Tagging the Cal group, since I think we'd like them to weigh in on when they might have availability to recalibrate using this new filter.
We have implemented this filter today and a calibration sweep was run with the filter on to determine the changes to the calibration.
The new filter is in FM8 of DARM2, and will be engaged in lownoise length control along with another DARM res g that is engaged.
I accepted the SDF in observe and loaded the guardian.