Tue Dec 12 10:08:21 2023 INFO: Fill completed in 8min 17secs
Gerardo confirmed a good fill curbside.
This morning at 8:15am (16:15UTC) I turned off the external power supply that powered both ITM HWS CCD cameras (Dalsa 1M60), located on the floor under the HWS table. We'll plan to check the magnetometer data to see if the 74738 comb is still present.
After we stopped HWS code and remotely turned off camera's in 74738, the DARM comb still remained. Dan, Daniel, Nutsinee expect this could be the framegrabber in the HWS computer still sending a 7Hz signal. We can troubleshoot this and how the camera's are grounded to the optics table later.
Looks like the comb is gone in the magnetometer channel. Figure 1 pre, figure 2 post.
Minor update: appending an hour-long "post" spectrum just to confirm & provide better comparison with previous 1-hour plots.
Camilla says the camera was turned on but not initialized at 10:03 Pacific / 18:03 UTC. Took a 30m spectrum starting 18:05 utc, attached. There is a strong near-1Hz comb and a strong near-57 Hz comb.
At 20:40UTC (12:40PT) we powered off both ITM computers h1hwsmsr and h1hwsmsr1. Dave has been notified. In 30 minutes we can try power cycling the cameras to see if the computers are off whether they re-initalize causing a comb.
Combs are still there with ITM computers off. Looks pretty much identical to the previous spectrum (very tiny frequency shift in the near-1Hz comb, though).
At 21:30UTC Erik unplugged the fiber connections that run from the back of the computers to the HWS cameras. Computers still off.
At 21:37UTC I power cycled the external supply to the cameras, to test if the cameras still turn on with a comb with no computer/frame grabber connection.
When the camera's restanted with the fiber link to the computers disconneted, we again saw the 1Hz and 57Hz combs, plot attached. It appears default for ITMX is 57Hz and ITMY is 1Hz. We should check if the camera software allows us to to put these to zero/ off.
For now we've replugged in the fiber connections, turned on the computers and restarted the hws code, both at 7Hz (this created some sdf diffs from the H1:TCS-ITMY_HWS_{}_POS_{X,Y} channels restarting with default values that kicked us out of observing, sorry)
Lasers still off, will turn lasers on tomorrow during commissioning to avoid sdf diffs.
Turned on both ITM SLEDs at 21:08UTC. These H1:TCS-ITM{X,Y}_HWS_SLEDSHUTDOWN channels should be added to sdf.
TITLE: 12/12 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Preventative Maintenance
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
SEI_ENV state: MAINTENANCE
Wind: 7mph Gusts, 5mph 5min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.27 μm/s
QUICK SUMMARY:
H1 has just lost lock after 32 hours following a commissioning test. Maintenance activities have begun.
Workstations were updated and rebooted. These were OS updates. Conda packages were not updated.
TITLE: 12/12 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 154Mpc
INCOMING OPERATOR: Corey
SHIFT SUMMARY: We're Observing and have now been Locked for 24 hours exactly. Quiet shift overall with just a few small earthquakes. **SEI_ENV and SEI_CONF have not been reloaded yet since we did not lose lock.
LOG:
00:00UTC Detector Observing and Locked for 16 hours
00:57 GRB-Short E458060
03:35 GRB-Short E458068
- NOT_GRB
06:10 Earthquake mode activated due to incoming earthquake from Samoa
06:20 Back to CALM
Observing at 159Mpc and have been Locked for 20 hours. Nothing to report.
Artem, Gabriele, Sheila, Louis Plots and Jupyter notebook with these results are also available here. The offloading scheme, in terms of actuation on different stages (UIM, PUM, TST), is different for LHO and LLO. We suspect that this could be the reason for apparent ESD noise coupling resulting in excess Low-frequency noise at LHO (see 74315). Here I present a comparison of external disturbance (CAL-DELTAL_CTRL_UIM/PUM/TST_DBL_DQ) and DARM error signal (OMC-DCPD-SUM_OUT_DQ) for LHO vs LLO. What we see is that external disturbance is very similar for both sites, while error signal is notably different, i.e. there are indeed significant differences in offloading scheme. There will be further discussion from Louis and my plots here are supporting material for that.
TITLE: 12/12 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 157Mpc
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 10mph Gusts, 8mph 5min avg
Primary useism: 0.03 μm/s
Secondary useism: 0.30 μm/s
QUICK SUMMARY:
In Observing and Locked for 16 hours. useism is starting to head back up.
TITLE: 12/11 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 156Mpc
INCOMING OPERATOR: Oli
SHIFT SUMMARY: Not much
Dave restarted the MC3 camera server and brought back said camera, that has been blue over the weekend at 17:04UTC.
LOG:
| Start Time | System | Name | Location | Lazer_Haz | Task | Time End |
|---|---|---|---|---|---|---|
| 16:51 | FAC | Karen | Optics lab | N | Tech clean | 17:08 |
| 17:16 | FAC | Karen | MidY | N | Tech Clean | 17:55 |
| 18:07 | SQZ | Nutsinee | Optics lab | N | Parts search | 18:23 |
| 18:48 | FAC | Kim, Karen | Clean storage area by CER | N | Inventory | 19:28 |
| 19:07 | TCS | Camilla | Optics lab | N | Tool investigation | 19:37 |
| 21:03 | VAC | Janos | Mids | N | VAC checks | 21:26 |
| 21:39 | VAC | Travis | Optics lab | N | Look for a hotplate | 21:45 |
I've taken the SUS_CHARGE node to DOWN so that the automatic in-lock charge measurements don't run before tomorrow morning's Tuesday maintenance time (and I can confirm that doing this does not take H1 out of observing). To re-enable the automated in-lock charge measurements, the SUS_CHARGE node should be requested to INJECTIONS_COMPLETE before next Tuesday.
We requested INJECTIONS_COMPLETE so this node is back in it's nominal configuration and will run injections next Tuesday at 7:45am.
This is a big-picture overview of combs in recent months.
"Relative priority" is a new metric documented in G2302324 (Athena Baches, Abby Wang, and Nick Baker). It is not finalized-- in fact these values differ slightly from the version in the linked presentation due to bug fixes-- and should be understood as (a) having arbitrary units, (b) reflecting the general importance of spectra artifacts to CW searches, and (b) being useful only for comparing artifacts in similar data sets-- for example, it should not be used to compare artifacts from monthly spectra and ones from weekly spectra. In this case it is used to compare and rank combs across monthly spectra.
Monthly August, October, and November Fscan spectra and their auto-generated lines lists were used. I skipped September because I still need to troubleshoot some issues with the lines list for that month.
Color-coding and letter labels convey identical information.
| A | |
| B |
1.66 Hz comb related to OM2 heater driver and variations which appeared during testing in October. Now mitigated. (72967)
|
| C | |
| D |
near-30 Hz comb. Cause unknown.
|
| E |
set of near-100Hz combs with various offsets. Cause unknown
|
| no tag |
Entries that are not persistent over multiple months.
|
| Ranking | Comb spacing | Comb offset | Relative priority | Label |
| 1 | 9.47425 | 0 | 46 | A |
| 2 | 9.47539 | 0 | 45 | A |
| 3 | 1.6611234 | 0 | 44 | B |
| 4 | 9.480545 | 9.48 | 39 | A |
| 5 | 4.9842174 | 0 | 22 | C |
| 6 | 29.969515 | 0 | 17 | D |
| 7 | 99.998456 | 70.029 | 5 | E |
| 8 | 6.64457 | 1.659 | 5 | |
| 9 | 99.99846 | 29.9695 | 5 | E |
| 10 | 99.99864 | 0 | 5 | E |
| 11 | 99.99844 | 89.909 | 2 | E |
| 12 | 99.99846 | 0 | 2 | E |
| 13 | 199.9969 | 10.09 | 1 | E |
| Ranking | Comb spacing | Comb offset | Relative priority | Label |
| 1 | 9.475416 | 0 | 43 | A |
| 2 | 9.474265 | 0 | 43 | A |
| 3 | 9.48057 | 0 | 42 | A |
| 4 | 9.480574 | 0 | 39 | A |
| 5 | 0.996787 | 0 | 32 | C |
| 6 | 4.98425 | 0 | 30 | C |
| 7 | 1.10864403 | 0 | 23 | B |
| 8 | 29.969514 | 0 | 15 | D |
| 9 | 1 | 0 | 6 | |
| 10 | 99.99846 | 29.9694 | 5 | E |
| 11 | 99.998644 | 0 | 5 | E |
| 12 | 99.99846 | 70.029 | 4 | E |
| 13 | 99.99845 | 10.09 | 2 | E |
| 14 | 99.99845 | 0 | 2 | E |
| Ranking | Comb spacing | Comb offset | Relative priority | Label |
| 1 | 9.47429 | 0 | 47 | A |
| 2 | 9.4754423 | 0 | 34 | A |
| 3 | 9.480612 | 0.0003 | 34 | A |
| 4 | 9.475456 | 9.4748 | 33 | A |
| 5 | 0.996785 | 0 | 32 | C |
| 6 | 4.984252 | 0 | 29 | C |
| 7 | 29.96951 | 0 | 15 | D |
| 8 | 28.44192 | 9.48 | 8 | |
| 9 | 0.12332754 | 0.1166 | 8 | |
| 10 | 0.1233334 | 0.0614 | 7 | |
| 11 | 47.4031 | 28.4417 | 6 | |
| 12 | 1 | 0 | 6 | |
| 13 | 99.998458 | 70.029 | 4 | E |
| 14 | 99.99846 | 29.9694 | 4 | E |
| 15 | 99.99844 | 0 | 2 | E |
| 16 | 99.99844 | 89.9088 | 2 | E |
| 17 | 8.426278 | 5.5594 | 1 |
FAMIS 20006
Between roughly 6 and 4 days ago, the regular temperature spikes seen in the control boxes, diode room, and chiller room readbacks stopped, meaning the temperatures there were more stable for that time. This does not seem to have had an effect on laser perfomance that I can see. I have yet to find a cause of this; maybe a fan in the chiller/diode room area was changed? Whatever changed, it was undone ~4 days ago and the behavior of these temperature channels has returned to normal.
The PMC temp output jumped up by about 0.3 about 3 days ago; unsure yet as to why.
Mon Dec 11 10:04:58 2023 INFO: Fill completed in 4min 55secs
Travis confirmed a good fill curbside.
Full Week DQ report is written here: DQ12/4-12/10
Notes:
Thank you for the opportunity as the shifter
The MC3 camera image has been down since 12:10 Fri 08dec2023 PST. There was nothing wrong with the camera, it just needed a restart of the h1cam12 server process on h1digivideo1. I killed the process and let monit restart a new one.
TJ, Ansel, Dave, Camilla
After finding that 1Hz and 5Hz DARM combs were caused by HWS camera sync frequency in 74617, we've turned off all HWS at 21:00UTC so we can check that nothing else from HWS is effecting DARM.
Stopped all HWS code (ETMY was already off 73915). Tuned off all cameras and RCX CLinks. Turned off the ITM SLEDs, ETMX laser will still be on but was tested in 69648 not to effect DARM.
As long as the computers aren't restarted I think that the H1:TCS-E{OPTIC}_HWS_ channels will remain reading their last values, if not Dave might need to "green-up" the CDS overview.
I temporarily took out the DIAG_MAIN notifications for all HWS. We will have to remember to put them back in though when we are done turning them on and off.
Unfortunately the 7 Hz comb is still present in DARM (figure 1) and in the magnetometer channel where we were tracking it (figure 2).
Extra: just to check that this really is a 7 Hz comb that started after the camera sync frequencies were moved to 7 Hz on Dec 5 (74617), and not something pre-existing, I re-plotted the magnetometer data for the time period when both were set to 5 Hz. This time period avoids any obscuring effects of the 1 Hz comb in magnetometer data. (The 1 Hz comb is much weaker than 7 Hz in DARM so it shouldn't really be an issue, but in the magnetometer data both are clear and there's a little more ambiguity there-- and anyway it's easy to check.) Indeed, no sign of 7 Hz during that hour except at frequencies that are also 5 Hz multiples (figure 3).
I put modified HWS() notifications back into DIAG_MAIN now Dave has a running script that turns the HWS camera/code off within 60 seconds of ISC_LOCK reaching state 580 and on within 60s of IFO being DOWN 74951. DIAG_MAIN may give the warning 'HWS code stopped for ITMX / HWS code stopped for ITMY' as soon as we loose lock for up to 60s, if this is annoying, we can make DIAG_MAIN cleverer. svn link
First observed as a persistent mis-calibration in systematic error monitoring Pcal lines which measure PCAL / GDS-CALIB_STRAIN affecting both LLO and LHO, [LLO Link] [LHO Link], characterised by these measurements consistently disagreeing with the uncertainty envelope.
It us presently understood that this arises from bugs in the code producing the GDS FIR filters there exists a sizeable discrepancy, which Joseph Betzwieser is spear-heading a thorough investigation to correct,
I make a direct measurement of this systematic error by dividing CAL-DARM_ERR_DBL_DQ / GDS-CALIB_STRAIN , where the numerator is further corrected for kappa values of the sensing, cavity pole, and the 3 actuation stages (GDS does the same corrections internally). This gives a transfer function of the difference induced from errors in the GDS filters.
Attached in this aLog, and its sibling aLog in LLO, is this measurement in blue, the PCAL / GDS-CALIB_STRAIN measurement in orange, and the smoothed uncertainty correction vector in red. Attached also is a text file of this uncertainty correction for application in pyDARM to produce the final uncertainty, in the format of [Frequency, Real, Imaginary].
After applying this error TF, the uncertainty budget seems to agree with monitoring results (attached).
After running the command documented in alog 70666, I've plotted the monitoring results on top of the manually corrected uncertainty estimate (see attached). They agree quite well.
The command is:
python ~cal/src/CalMonitor/bin/calunc_consistency_monitor --scald-config ~cal/src/CalMonitor/config/scald_config.yml --cal-consistency-config ~cal/src/CalMonitor/config/calunc_consistency_configs_H1.ini --start-time 1374612632 --end-time 1374616232 --uncertainty-file /home/ling.sun/public_html/calibration_uncertainty_H1_1374612632.txt --output-dir /home/ling.sun/public_html/
The uncertainty is estimated at 1374612632 (span 2 min around this time). The monitoring data are collected from 1374612632 to 1374616232 (span an hour).
J. Kissel, J. Betzwieser
FYI: The time at which Vlad used to gather TDCFs to update the *modeled* response function at the reference time (R, in the numerator of the plots) is
2023-07-27 05:03:20 UTC
2023-07-26 22:03:20 PDT
GPS 1374469418
This is a time when the IFO was well thermalized.
The values used for the TDCFs at this time were
\kappa_C = 0.97764456
f_CC = 444.32712 Hz
\kappa_U = 1.0043616
\kappa_P = 0.9995768
\kappa_T = 1.0401824
The *measured* response function (GDS/DARM_ERR, the denominator in the plots) is from data with the same start time, 2023-07-27 05:03:20 UTC, over a duration of 384 seconds (8 averages of 48 second FFTs).
Note these TDCF values list above are the CAL-CS computed TDCFs, not the GDS computed TDCFs. They're the value exactly at 2023-07-27 05:03:20 UTC, with no attempt to average further over the duration of the *measurement*. See attached .pdf which shows the previous 5 minutes and the next 20 minutes. From this you can see that GDS was computing essentially the same thing as CALCS -- except for \kappa_U, which we know
- is bad during that time (LHO:72812), and
- unimpactful w.r.t. the overall calibration.
So the fact that
:: the GDS calculation is frozen and
:: the CALCS calculation is noisy, but is quite close to the frozen GDS value is coincidental, even though
:: the ~25 minute mean of the CALCS is actually around ~0.98 rather than the instantaneous value of 1.019
is inconsequential to Vlad's conclusions.
I'm adding the modeled correction due to the missing 3.2 kHz pole here as a text file. I plotted a comparison showing Vlad's fit (green), the modeled correction evaluated on the same frequency vector as Vlad (orange), and the modeled correction evaluated using a dense frequency spacing (blue), see eta_3p2khz_correction.png. The denser frequency spacing recovers error of about 2% between 400 Hz and 600 Hz. Otherwise, the coarsely evaluated modeled correction seems to do quite well.
The above error was fixed in the model at GPS time 1375488918 (Tue Aug 08 00:15:00 UTC 2023) (see LHO:72135)
