Jonathan, Dave:
The file copy completed yesterday. Today we reconfigured nds0 to serve the past 6 months of raw minute trends from its new permanent location. I restarted nds0 daqd at 13:23 PST when H1 was out of lock.
The deletion of the files from tw0's SSD-RAID took from 14:18 to 16:28 (2hr 10min). Final cleanup was to delete the temporary directories under /rawtrends/minute_raw_deleteme on tw0.
TITLE: 12/15 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 160Mpc
OUTGOING OPERATOR: Ibrahim
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 2mph Gusts, 1mph 5min avg
Primary useism: 0.03 μm/s
Secondary useism: 0.23 μm/s
QUICK SUMMARY:
Currently H1 is Locked in NOMINAL_LOW_NOISE and OBSERVING.
TITLE: 12/14 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Commissioning
INCOMING OPERATOR: Tony
SHIFT SUMMARY:
IFO is in NLN and COMMISSIONING
17:56:40: Range Loss Investigation
(4 Screenshots)
20:45 UTC - Started running calibration measurements (1 screenshot)
20:50 UTC - Broadband calibration measurement started
21:18- UTC - Next calibration (simuLines) started.
21:19 - UTC - lost lock (alog 74814)
21:20 UTC - Relocking
21:27 UTC - Nds0 daqd restarted by Dave right after lockloss - caused all ndscopes to stall and require resetting.
22:37 UTC - Air handler red alarm started blaring. Eric was the last person in there earlier to check on these. I called him and he’s investigating. Trends show that it was off for about 15 minutes. Alarm is back to normal as of 14:50 UTC.
23:08 - Reached NLN - Resumed commissioning.
LOG:
Start Time | System | Name | Location | Lazer_Haz | Task | Time End |
---|---|---|---|---|---|---|
22:11 | FAC | Eric | CS | N | Check air handlers | 22:21 |
19:45 | FAC | Kim | LVEA External Rollup Door | N | Cardboard Haul | 20:45 |
19:45 | FAC | Karen | Woodshop | N | Technical Cleaning | 19:55 |
18:38 | FAC | Karen | Vac Prep | N | Technical Cleaning | 19:38 |
17:41 | FAC | Cindi | Wood shop area | N | Laundry | 18:41 |
17:42 | Nutsinee | Optics Lab | Local | Testing | 18:38 | |
16:43 | FAC | Karen, Kim, Randy | Garb Storage | N | Assessing inventory | 16:43 |
With different iteration than yesterday of DARM offloading configuration, I checked MICH with the current MICHFF FM8 orange and the MICH FF fitted from yesterday's data FM9 black 74791, plot attached. Surprising that in our 20-70Hz band the new FF was worse. Maybe as the data was taken when the IFO was thermalizing yesterday (2h30) rather than today after 14hr in lock.
Retook the MICH excitations at ~21:20UTC using FM8 and fit using Gabriele's InteractiveFitting GUI. Results attached. Loaded this fit into FM1 as '12-14-23' ready for when we next use the new DARM filters. Lost lock before we could test the new fit.
21:18- UTC - Calibration Measurement (simuLines) started.
21:19 - UTC - Lost Lock
Potentially (but unlikely) related - Oli also lost lock during Simulines during their shift - alog 74716
Relocking Now as of 21:20 UTC.
Attached shows that the sus excitations began shortly before the oscillation begain, but that the DARM1 and SWEPTSINE excitations began after the oscillation. Investigation still ongoing as to what caused the lockloss though.
We plotted the fast 16kHz channel of H1:SUS-ETMX_L2_CAL_EXC_OUT_DQ and can see this excitation caused the ETMX oscillation and lockloss. Sheila/Jenne state this is not that surprising as the gains have been changed the ETMX filters. Plot attached.
IFO is in NLN and OBSERVING. We are about to transition into planned comissioning for the next 3 hours (20:00 UTC - 23:00 UTC).
Thu Dec 14 10:09:22 2023 INFO: Fill completed in 9min 18secs
Travis confirmed a good fill curbside.
TITLE: 12/14 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 156Mpc
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 4mph Gusts, 3mph 5min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.22 μm/s
QUICK SUMMARY:
IFO is in NLN and OBSERVING
We were kicked out of observing due to the BS camera not updating and the camera servo going back onto ADS. The ITMX camera also seems to be reading only 0's. Dave is currently working to fix this.
BS camera (h1cam26) is back online. I had to power cycle the camera via sw-lvea-aux POE port off/on sequence and then restart the camera process on h1digivideo2.
H1 is back in observe.
All fixed. Out of Observing from 0841-0933UTC
Plot shows second trends from 00:40 to 01:35 PST this morning. The top trace (blue) is BS cam x-centroid, the middle trace (green) is H1-range and the lower trace (orange) is H1-observe-bit with x100 gain.
H1 kept a good range throughout apart from a momentary drop to 20MPc.
H1 came back to observe within a few minutes of BS camera image restoration.
FRS29964 has been opened for this issue.
Louis, Sheila, Jenne
Today we tried a new DARM loop configuration. We are not implementing this for observing right now, we would need to retune feedforward and redo calibration before we could do that.
There is a new guardian state in ISC_LOCK, called NLN_ETMY. This state turns off feedforward and transitions DARM control to ETMY (L3,L2, L1), it can be used when we are in nominal low noise (but not necessarily from any states before LOWNOISE_ESD_ETMX). This was tested Tuesday morning and today, so it has worked twice.
We set up a new configuration of filters for ETMX. Louis has been using pyDARM to model the DARM crossovers, and we will add more details about these new filters in a later alog. The first attached screenshot shows the configuration that we first transitioned to, without the new boost. This keeps the overall loop similar to our earlier configuration, the main differences are:
Measurements:
The last attachment shows spectra of the DAC counts for ETMX, and DCPD SUM. The new configuration with the boost on reduces the RMS counts on the ESD by a little more than a factor of 2 compared to the configuration we've been using (most of the improvement is from the additional boost). The new configuration also supresses DCPD sum more below 10Hz.
Camilla looked at the MICH FF with the new configuration and saw that it needs to be retuned, she took measurements that can be used to do that.
We lost lock as I was trying to semi-manually transition DARM control back to ETMY from this new configuration.
Here's the DARM1 and DARM2 banks that were used for the test. DARM1 FM1 is off
Gabriele's boost design string is
zpk([-29.9198+i40.0369;-29.9198-i40.0369],[-18.297+i23.0198;-18.297-i23.0198],1)
It's plotted in boost_filter.png in blue. The red trace is the 4.5kHz boost we want to replace. The new filter is meant to be less aggressive in magnitude and cost less in phase.
Here's a quick plot of the DARM loop model with today's test changes in place. Zoomed out. Zoomed in. The OLG measurements Sheila took match quite well. The PUM measurements still don't. And they suggest we're over estimated our phase margin at the PUM stage in the model. Discussion to follow.
I'm hoping that it was just incidental and won't come back, but I would like for us to watch for increases in the bounce and roll modes (and maybe other peaks). They seemed a bit higher on the DARM FOM yesterday, and they also look a bit elevated in the plots in this alog thread (but that was all from the same time, so just a confirmation of what I thought I was seeing on the wall). If this new configuration (which gives us so many improvements) exacerbates those modes, we may have to remember to turn on some active damping for them.
I've set up a new Guardian state inISC_LOCK
called DARM_RECOVER. This state is meant to be run from Sheila'sNLN_ETMY
to bring the IFO back into the nominal DARM loop configuration. I will use test it after tomorrow morning's set of high-risk tests just before maintenance day.
After our clipping saga and moving PR3 to un-clip, we wondered where are we on PR2. The spot on PR3 should always be in the same position lock-to-lock, because it is in roughly the same Gouy phase as the ITMs, and we servo to the spot position on the ITMs. However, the pointing of PR3 is free to drift, which would make the spot on PR2 (and PRM I suppose) be free to be different. The pointing of PR2, PRM, and IM4 would follow any such PR3 drift to ensure that the cavity axis is normal to the PRM, since that's a requirement for resonance. This indicates that the spot position on PR2 could be a witness of sorts for PR3 pointing, since we seem to not be able to trust either the top mass PR3 OSEMs or the PR3 oplev over long periods.
I found that Gabriele and Elenna had most recently checked the spot position on PR2 in April 2023 (alog 69025) and minimized the lines in PRCL_IN1. The PR2 A2L gains have not changed since that time (until I temporarily changed them today). I'll note that, just to make it easy to re-find the link, around that time there was also some PR3 dithering done in alog 69026.
I dithered PR2 in yaw at 7.9 Hz, and looked at the peaks in both PRCL_IN1 and DARM_IN1. Just in case it matters, this first measurement was taken *before* Sheila changed the DARM offloading, but with all LSC FF off, and also quite early in the lock (so not fully thermalized). Somewhat confusingly, the yaw line heights seem to do *different things* in DARM vs PRCL when I change the Y2L gain. In the first attached screenshot, I've changed the Y2L gain from nominal -7.4 (blue trace in DTT) to further-from-center -7.7 (red trace in DTT), and the peak in PRCL (top panel of DTT) has increased while the peak in DARM (lower panel of DTT) has gone down. I would think that PRCL is the thing to trust, since that sould be a more direct measure of A2L coupling. But, overall, we probably want to be minimizing the effect in DARM (although, we don't really care in DARM at this low a frequency). Perhaps this is somewhat similar to the phenomenon we've seen where A2L gains for the test masses have to be set for DARM around 20 Hz to be effective, since the A2L values one would chose by looking at DARM below 10 Hz aren't good.
In the second attachment, I instead make a Y2L change when dithering PR2 at 24.1 Hz, and (moving PR2's Y2L gain closer to zero) both PRCL and DARM agree that moving closer to center of the optic makes the peak go down slightly. Also notable though is that we already seemed to be fairly close to the minimum peak height I could get with making Y2L step sizes of 0.1 for PR2. This means that the best place for the Y2L gain of PR2 has only changed by about 0.2 from April 2023 to Dec 2023. I'll have to look up what that means in mm, but it's going to be small. We don't have a measurement of where it was at any time between those times though, and in particular we don't know what value of A2L would minimize the coupling during the time when we had clipping issues.
In the third attachment, I had put the Y2L gain back to its nominal value of -7.4, and instead change the P2L value. Even though this measurement is at about 25 Hz, PRCL and DARM's responses are back to being opposite of each other. There was a dramatic lowering of the peak height in PRCL, but the peak got a bit bigger in DARM. Again, overall not a big change though.
Next steps: re-find the conversion between A2L units and mm for triples like PR2. Potentially could try small move of PR3, to see what is the effect on optimal A2L values, to see if PR2's A2L values can be used even as a rough reference for PR3 pointing.
I did a little more of this today, while LLO was relocking.
One thing I noticed was that today while we were early in the lock (so still thermalizing), I could make pretty dramatic changes in the peak height of the pitch PR2 dither peak in PRCL, but there was very little change in the peak in DARM. I'm not really sure what that means, but PRCL is certainly where I expected it to show up more strongly, so maybe that's fine.
While dithering PR2 in pitch at ~25 Hz, I found that the optimal P2L value was again about -0.310 (same as yesterday). I then moved PR3 by about 0.5 urad in pitch, in steps of 0.03 urad, and tried to re-optimize the PR2 P2L gain, and found that it was more optimal at about -0.36 (when taking P2L steps of 0.05). So, there is a bit of an effect when moving PR3, although I can't promise that the effect was due to thermalization or something else.
When finished, I put the PR2 A2L values back to their nominals, and walked PR3 back to its nominal slider position. I'll note that I didn't see any glitches or anything 'bad' when moving PR3 in steps of 0.03 (which is already 3x larger than I was doing in Observe last week), so we can probably take slightly larger steps next time we do this, to make things go faster.
New MICHFF measurements with the new DARM offloading on were taken at at 23:20UTC after 2h30 in NLN. Saved in /opt/rtcds/userapps/release/lsc/h1/scripts/feedforward
Followed the README in the same folder and used LSC_FF_PrepareData.ipynb to create mich_ff_tofit.txt, plan to fit this tomorrow morning so we can try new MICHFF during commissioning when we next turn on the new DARM offloading.
Gabriele and I fit this data using his InteractiveFitting GUI. Results attached.
Attached is the change to MICH with the different DARM filters 74790 (black) verus how it wa with nominal DARM last week 74642 (pink).
Sheila, Elenna, Camilla. Last done in August 72140. Doing now as BRUCO shows ASC coherences 74664 and want to understand current ASC coupling beofre OM2 commissioning.
Saved in /ligo/gitcommon/NoiseBudget/aligoNB/aligoNB/H1/couplings/
The new CHARD_Y quiet time reference of H1:ASC-CHARD_Y_SM was very different to the old quiet time reference: compare new (red) to 10th August (blue) of bottom left plot attached.
Instructions: (Naoki put laser noise injection instructions in 70642)
Continued with these injections:
Commited to git as under [master 3fcc1f5]
Naoki ran the analysis, plots in 74788.