H1 has been locked for 2hrs with a range just under 165Mpc. This is with microseism above the 95th percentile just under 1um/s.
After Austin started an Initial Alignment, locking looked optimistic, but then several consecutive locklosses later, I ended up transitioning SEI_CONF node from USEISM --back to--> WINDY. This required:
The picket fence has been pretty noisy and flashes yellow for various channels (don't recall seeing this earlier this week).
Violins are just under an order of magnitude higher than they were last night at 3e10^-17.
TITLE: 01/05 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Aligning
OUTGOING OPERATOR: Austin
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 5mph Gusts, 3mph 5min avg
Primary useism: 0.15 μm/s
Secondary useism: 0.55 μm/s
QUICK SUMMARY:
Arrived to my shift Mid-Initial Alignment for H1 per Austin's recovery from commissioning lockloss (and due to minor hardship for getting through early steps of ISC-LOCKing). Status: Alignment complete and H1 has just passed the DRMI locking steps. NOTE: Lockloss at LOW NOISE COIL DRIVERS, & then CHECK AS SHUTTERS.
Austin also mentioned the unique low-freq seismic activity--Microseism was starting a slow decrease 24hrs ago, but in the last 12hrs it started to turn around and is currently back up to where it was 24hrs ago (which is at/above the 95th percentile). BUT--a unique observation is that the "secondary" microseism band has also increased around the 12hr turnaround. So because of the odd microseism and rough locking in early states, SEI_CONF is at the USEISM state. If we want to return SEI_CONF to WINDY, It's not a simple transition to WINDY. TJ shared how we make this transition:
For much of the above---it most likely not be done tonight, unless the Earth becomes much more quieter seismically for us.
Dave, Erik, Camilla, Jonathan, During a lock loss Jonathan went to EY and got the network interface configured. This was enough to get the machine back on the network. When the machine was replaced yesterday the camera was plugged into the wrong slot on the pci board, this was fixed as well. Dave, Erik, and Camilla remotely activated and tested the camera. Some notes: * During an iteration of work inside, go outside for cell phone reception something in the door mechanism to EY got stuck and it would not open despite being set to unlocked by the operator and by a key fob. After many tries it eventually opened. * Since the re-keying there is no key in the lock box.
At 15:30:48 PST this afternoon h1hwsex crashed again. It had crashed Sat 23 Dec 2023 and was rebooted this Tue 02 Jan 2024. It ran for just about 2 days before crashing again.
Camilla says it is OK to leave the computer off overnight, Erik will replace this computer with a V1 during tomorrow's commissioning break (Fri 05 Jan 2024).
H1 was unlocked at the time of the crash, which means the HWS camera is acquiring images. Camilla confirmed it is OK to leave the camera in this state, it is the ITM cameras which have been causing noise issues.
To green up EDC and SDF, I started the HWS EX dummy IOC on cdsioc0. This is running as user 'ioc' in a tmux session.
TITLE: 01/04 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Lock Aquisition
INCOMING OPERATOR: Corey
SHIFT SUMMARY:
- 16:17 - GRB short E463149
- 16:51 - Superevent S240104b
- 17:11 - 18:10 - Fire pump testing began - Tagging DetChar in case this shows up as noise on their end
- EX saturation @ 22:02
- 22:35 - went into COMMISSIONING for opportunistic DARM measurements while LLO is down due to microseism
- ISC went through MICH FRINGES a few times without being able to move up, which is my cue to begin running an IA, which is currrently ongoing
- Mystery rise and steady hold of primary microseism is still apparent, coupled with a now rising secondary microseism, I have moved the SEI CONF guardian to USEISM to help make locking less cumbersome
LOG:
Start Time | System | Name | Location | Lazer_Haz | Task | Time End |
---|---|---|---|---|---|---|
16:50 | FAC | Johnson Controls | Site | N | Fire panel work | ?? |
17:03 | FAC | Karen | Wood shop | N | Tech clean | 17:20 |
18:26 | FAC | Kim | H2 | N | Tech clean | 18:30 |
19:33 | VAC | Janos | MX | N | Vac checks | 19:45 |
21:26 | FAC | Randy | EX Mech | N | Inventory | 21:56 |
22:34 | VAC | Gerardo | MX | N | Check purge air | ?? |
23:09 | CDS | Jonathan | EY | N | Finish set up of HWS computer | 23:59 |
Sheila, Louis
We tried transitioning to the new DARM configuration today slowly to debug the transition from NEW_DARM -> [nominal] DARM. We've had plenty of success transitioning into the new DARM state recently using the ETMY_NLN and NEW_DARM ISC_LOCK guardian states. However, today we encountered several errors when doing so. Some syntax errors cropped up from a few lines in ETMY_NLN (that neither of us recognized) that kept us from moving beyond the main() method. We ran the run() instructions by hand (with the GRD state back in NLN) and fixed the syntax errors in ETMY_NLN and then continued by hand to the NEW_DARM configuration (while the guardian state stayed in NLN) but lost lock when swapping the gains on SUS-ETM{Y,X}_L3_LOCK_L
. We confirmed that all the expected DARM loop filters were installed and engaged as expected before moving to ETMX (NEW_DARM) from ETMY. It's not clear to us yet why we lost lock this time around after having virtually no issues switching to the new DARM state recently.
Lockloss @ 22:57 - caused by commissioner measurement.
H1 is still locked, currently at 44.5 hours. All systems appear stable, though primary microseism is currently on the rise - cause of it is unknown.
Thu Jan 04 10:07:02 2024 INFO: Fill completed in 6min 58secs
Gerardo confirmed a good fill.
BSC high freq noise is elevated for these sensor(s)!!!
ITMX_ST2_CPSINF_H1
ITMX_ST2_CPSINF_V1
The Primary Microseism is currently elevated in a strange way, which may explain a slight increase in the noise Spectra across all chambers. But other than that I did not see anything that looked wildly different from the last ISI CPS Noise Spectra Check referenced in alog 75088.
TITLE: 01/04 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 157Mpc
OUTGOING OPERATOR: Tony
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 5mph Gusts, 3mph 5min avg
Primary useism: 0.11 μm/s
Secondary useism: 0.36 μm/s
QUICK SUMMARY:
- H1 just hit a 40 hour lock and appears stable
- CDS/DMs ok
- EQ band looks to be slowly on the rise but still within a region in which we can operate
TITLE: 01/04 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 156Mpc
INCOMING OPERATOR: Tony
SHIFT SUMMARY:
Nice shift with H1 locked for 32hrs and H1/L1 double coincident the entire shift. Warnings of earthquakes, but nothing noticeable in the control room. Winds slowly tapered down as the shift went on and all else is well.
TITLE: 01/04 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 161Mpc
OUTGOING OPERATOR: Austin
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 5mph Gusts, 4mph 5min avg
Primary useism: 0.04 μm/s
Secondary useism: 0.58 μm/s
QUICK SUMMARY:
H1's just passed the 24hr lock mark w/ a range centered at 160Mpc. We are running without our pair of large central TV monitors (staging to be replaced)! Note of no commissioning until Friday.
Smooth running shift with no issues to report and H1 rocking steady slightly above 160Mpc & 28hrs of lock!
Ibrahim, Oli, Betsy, Arnaud, Fil
Context: In December ('23) We were having issues confirming that the damping, OSEMs, electronics and model were working (or rather, which wasn't working).
I have more thorough details elsewhere but in short:
Eventually, we were able to go through Jeff and Oli's alog 74142. Here is what was found:
All "crude push around offsets" in the test bank yielded positive drives in the damp channels. These are the ndscope screenshots. Different offsets were needed to make the offset change more apparent in the motion (such as with L). A minimum of 1,000 was arbitrarily chosen and was usually enough.
Transfer Functions: where it gets interesting... (DTT Screenshots)
In these DTTs, each reference (black) are the transfer functions without the damping, while the red traces are with the damping.
All "translation" degrees of freedom (L, V, T) showed correct damping, peak location and resonance
All "rotation" degrees of freedom (P, R, Y) showed completely incorrect damping, usually showing shifted peaks to the right (higher freq).
In trying to figure out why this is, we asked:
(In)conclusion:
It seems that whenever the OSEMs push in the same direction, everything goes as planned, hence why all translation damping works. When we ask the OSEMs to push in opposing directions with respect to one another though, they seem to freak out. This seems to be the prime "discovery" of finally getting the transfer functions.
This is the "for now" update - will keep trying to find out why until expertise becomes available.
Rahul, Ibrahim, Austin
Context: After hopping off the TS call where we decided to try the Pitch TF again and reducing the Damping gain, I met with Rahul and Austin in the control room and we decided to check some more basic OSEM health first.
Something I forgot: When Oli and I were in the control room taking the transfer functions the first time around, I noticed that for the rotational degrees of freedom (P, Y, R), the OSEM outputs were railing immediately (both visibly in number and on the overflow page). I wondered whether I should re-do the TFs without the saturations, by empirically testing the gain until it doesn't overflow. I ultimately kept the nominal -1G in order to report the initial "this is how bad it is" results. This will become relevant later.
Rahul was concerned that the OSEM spectra for the OSEMs that are in M1 were too noisy so we took some spectra measurements of the OSEMs themsevles to see if this was the case ... and it was. These are the screenshots below. We tried them with and without damping to see if damping works, and it doesn't seem like the damping is working exactly as it should be. Additionally, the <10Hz noise is 1-2 orders of magnitude too high according to Rahul. This is a way more "up the chain" (down the chain?) issue and could result in the weirdness we're seeing at the TF level. Why is this?
The Plan
Following these quick checks - once I'm out of the Staging Building:
Minor tasks also include:
Updates incoming.
TITLE: 01/03 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 152Mpc
OUTGOING OPERATOR: Austin
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 6mph Gusts, 4mph 5min avg
Primary useism: 0.03 μm/s
Secondary useism: 0.46 μm/s
QUICK SUMMARY:
Austin handed off an H1 nearly at NLN. I eventually took H1 to Observe, at 0011utc, but at 0037, have been getting bumped out of OBSEVERING due to a gain change (i.e. H1:CAL-INJ_CW_GAIN) for CALINJ SDF.
Attached is the last 20+ min of these gain changes from 1.0 to 0.0 every ~1min. Have Louis on the phone now.
Forgot to mention that Jim chatted with me regarding the HAM3 ISI, and that if the glitching issues return, I should phone him if it is before ~8pm. Luckily, we have not had to deal with this for our current 24+hr lock!
STATE of H1: Observing at 154Mpc
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 8mph Gusts, 5mph 5min avg
Primary useism: 0.04 μm/s
Secondary useism: 0.94 μm/s
QUICK SUMMARY:
InLock Sus charge measurements likely caused the lockloss this morning. Though there was a PI message at the same time. but the PI didn't seem high enough to break a lock.
https://ldas-jobs.ligo-wa.caltech.edu/~lockloss/index.cgi?event=1387641172
Relocking started before 1600UTC.
16:45 UTC NOMINAL_LOW_NOISE reached and OBSERVING at 16:51 UTC
SQZ manager dropped us into commissioning at 17:22 UTC
Back to Observing at 17:24 UTC
N2 truck arrived at Y end around 17:34.
I missed the time that the N2 truck left. I believe it was shortly after 1900 UTC
The Temps in the in the VPW ranged from 66F to 73F. Im not sure what the correct range should be, but the one that read 73 was nearest the warmed server exhaust on that side of the room.
Attached is a plot of today's noisy events related to the LN2 delivery to the tank for CP7. Since the IFO was locked flagging the respective groups.
We lost lock during the SETUP step of ESD_EXC_ETMX, plot attached. ETMX_L3_DRIVEALIGN_L2L (bottom right) had an output but I don't think it should have as the feedback is on ITMX at this point and the excitation hadn't started yet. Should check this before next Tuesday.
ESD_EXC_ETMX log before lockloss:
2023-12-26_15:52:32.263999Z ESD_EXC_ETMX [SETUP.main] ezca: H1:SUS-ETMX_L3_DRIVEALIGN_L2L_TRAMP => 1
2023-12-26_15:52:32.264996Z ESD_EXC_ETMX [SETUP.main] ezca: H1:SUS-ETMX_L3_DRIVEALIGN_L2L_GAIN => 1
2023-12-26_15:52:33.394157Z ESD_EXC_ETMX [SETUP.main] ezca: H1:SUS-ETMX_L3_DRIVEALIGN_L2L_SW1S => 0
2023-12-26_15:52:33.645345Z ESD_EXC_ETMX [SETUP.main] ezca: H1:SUS-ETMX_L3_DRIVEALIGN_L2L => ONLY ON: OUTPUT, DECIMATION
Looking at a successful ETMX SETUP state, you can see that there is still a small time where ETMX_L3_DRIVEALIGN_L2L has an output, the IFO just survives the glitch. This happens when the gain is changed from 0 to 1 (to allow excitation through) but before the INPUT is turned off. I've swapped the order of these two lines and added a 1sec sleep between them to make sure that the input is turned off before the gain is ramped to 1. Edit has been saved and will reload during next commisioning period.
Jenne, Naoki, Louis, Camilla, Sheila
Here is comparison of the DARM CLEAN spectrum with OM2 hot vs cold. The second screenshot shows a time series of OM2 cooling off. The optical gain increased by 2%, as was seen in the past (for example 71087). Thermistor 1 shows that the thermal transient takes much longer (12 + hours) than what thermistor 2 says (2 hours).
Louis posted a comparison of the calibration between the two states, there are small differences in calibration ~1% (74913). While the DARM spectrum is worse below 25Hz, it is similar at 70 Hz where we in the past thought that the sensitivity was worse with OM2 cold. From 100-200 Hz the sensitivity seems slightly better with OM2 cold, some of the peaks are removed by Jenne's jitter subtraction (74879) but there also seems to be a lower level of noise between the peaks (which could be small enough to be a calibration issue). At high frequency the cold OM2 noise seems worse, this could be because of the squeezing. We plan to take data with some different squeezing angles tomorow and will check the squeezing angle as part of that.
So, it seems that this test gives us a different conculsion than the one we did in the spring/summer, and that now it seems that we should be able to run with OM2 cold to have better mode matching from the interferometer to the OMC. We may have not had our feedforwards well tuned in the previous test, or perhaps some other changes in the noise mean that the result is different now.
Is this additonal nosie at low frequency due to the same non-stationarity we oberved before and we believe is related to the ESD upconversion? Probably not, here's why.
First plot compares the strain spectrum from two times with cold and hot OM2. This confirms Sheila's observation.
The second and third plots are spectrograms of GDS-CALIB_STRAIN during the two periods. Both show non-stationry noise at low frequency. The third plot shows the strain spectrogram normalized to the median of the hot OM2 data: beside the non-stationariity, it looks like the background noise is higher below 30 Hz.
This is confirmed by looking at the BLRMS in the 16-60 Hz region for the two times, as shown in the fourth plot: its higher with cold OM2
Finally, the last plot shows the correlation between the ESD RMS and the strain BLRMS, normalized to the hot OM2 state. There is still a correlation, but it appear again that the cold OM2 state has an additional background noise: when the ESD RMS is att the lower end, the strain BLRMS setlles to higher values
Here is the same comparison, without squeezing. Using times from 74935 and 74834
This suggests that where cold OM2 seems better than hot OM2 above that is due to the squeezing (and the jitter subtraction Jenne added, which is also on in this plot for cold OM2 but not for hot OM2). And the additional noise with cold OM2 reaches up to about 45Hz.
After we optimized ADF demod phase in 74972, the BNS range seems better and consistently 160-165Mpc. The attached plot shows the comparison of OM2 cold/hot with/without SQZ. The OM2 cold with SQZ is measured after optimization of ADF demod phase and other measurements are same as Sheila's previous plots.
This plot supports what Sheila says in the previous alogs.
Echo-ing the above, and summarizing a look at OM2 with sqz in both Sept 2023 and Dec 2023 (running gps times dictionary is attached here).
If we compare the effect of squeezing -- there is higher kHz squeezing efficiency with hot OM2. We can look at either just the darm residuals dB[sqz/unsqz] (top), or do subtraction of non-quantum noise (bottom) which shows that hot OM2 improved the kHz squeezing level by ~0.5 dB at 1.7 kHz (the blue sqz blrms 5). This is consistent with summary pages: SQZ has not reached 4.5 dB since cooling OM2 74861. Possibly suggests better SQZ-OMC mode-matching with hot OM2.
Without squeezing, cold om2 has more optical gain and more low-freq non-quantum noise. Better IFO-OMC mode-matching with cold OM2.
In total, it's almost a wash for kHz sensitivity: heating OM2 loses a few % optical gain, but recovers 0.2-0.5 dB of shot noise squeezing.
It's worth noting the consistent range increases with SQZ tuning + improvements: even in FDS, there is a non-zero contribution of quantum noise down to almost 50 Hz. For example Naoki's adjustment of sqz angle setpoint on 12/21 74972 improved range, same for Camilla's Jan sqz tuning 75151. Looking at DARM (bottom green/purple traces), these sqz angle tunings reproducibly improved quantum noise between about 60-450 Hz.
Here are some more plots of the times that Vicky plotted above.
The first attachment is just a DARM comparison with all 4 no sqz times, OM2 cold vs hot in December vs September.
Comparing OM2 hot September vs December shows that our sensitivity at from 20-40 Hz has gotten worse since September, the MICH coherence seems lower while the jitter and SRCL coherence seem similar. The same comparison for OM2 cold shows that with OM2 cold our sensitivity has also gotten worse from 15-30 Hz.
Comparing cold vs hot, in September the MICH coherence did get worse from 60-80 Hz for cold OM2, which might explain the worse sensitivity in that region. The MICH coherence got better from 20-30 Hz where the sensitivty was better for cold OM2. The December test had better tuned MICH FF for both hot and cold OM2, so this is the better test of the impact of the curvature change.
As Gabriele pointed out with his BRUCO, 74886 there is extra coherence with DHARD Y for cold OM2 at the right frequencies to help explain the extra noise. There isn't much change in the HARD pitch coherence between these December times, but the last attachment here shows a comparison of the HARD Y coherences for hot and cold OM2 in December.
Peter asked if the difference in coherence with the HARD Yaw ASC was due to a change in the coupling or the control signal.
Here is a comparison of the control signals with OM2 hot and cold, they look very similar at the frequencies of the coherence.
Before the HWS ETMY IOC was started on h1hwsey, I stopped the simulation IOC which had been running on cdsioc0.
This was running under systemd control as h1hwsetmy_dummy_ioc.service. I have stopped and disabled this service.
h1tcshwssdf was restarted a few times this afternoon as ETMX and ETMY HWS IOCs came and went.
Camilla started the camera control code for ETMY.