Today during Manitenance I moved the analog camera on the HAM2 East Door from A2F1 to A2F4, to gain a closer view of AOE2, and the surrounding components. I also repositioned the camera within the camera housing, and changed the lens. Images attached later.
(Camilla, Corey)
Cranes Not In Historical Parking Spots
Left CER Sweep for Marc & Dick to handle since they are running RF measurements when H1 is locking.
The primary and redundant calibration pipelines producing DMT hoft were restarted around 11 am PST during maintenance after an email exchange with the calibration group, ldas, and Keita and the operator was informed. The restarts were to fix a problem with the DMT redundant hoft pipeline (it was wrting 1 s of data to disk every 3 seconds) and to reduce the latency of the DMT primary hoft pipeline (which had a latency of 17 s as measured at Caltech, and now its 9 s). The DMT calibration pipelines are both working again.
Found the FSS common gain set to 17.7; its usual setpoint is 20. Trending back, it looks like the gain changed at ~10:20 UTC (~2:20 PST) on December 1st, which coincides with Ed's RefCav locking issues from Sunday morning. I spoke with Ed (both this morning and during Sunday's RefCav locking problem) and he did not adjust the common gain during this time, only the fast; Guardian was controlling the common gain (recall that we use a Guardian node to automatically reduce and restore to the FSS common gain to cure loop oscillation issues). At this point the only guess I have as to why the common gain didn't come back to its usual value of 20 was some issue with the Guardian node (will talk to TJ). I restored the FSS common gain to 20 at ~19:58 UTC (11:58 PST). This channel is unmonitored in SDF (as Guardian changes it as previously mentioned to cure loop oscillations), so there was no SDF change to accept.
Edit: Added attachment showing common gain change on Sunday morning.
This morning I tweaked the beam alignment into the FSS RefCav; the TPD had dropped to ~2.8V. Unfortunately, I was only able to get the TPD back to ~3.4V by tweaking the beam alignment (instead of the 4.7V we had at the end of the last adjustment). We just had a storm system move through the area over the weekend (these drops seem to coincide with storms moving through), so I'll monitor the RefCav TPD throughout the week to see if it returns to normal. If it doesn't, then an enclosure incursion during the next maintenance window may be necessary to tweak the FSS alignment.
I reset both PSL power watchdogs at 17:40 UTC (9:40 PST). This completes FAMIS 10739.
Cheryl transitioned the LVEA to Laser Hazard
Ops Shift Transition: 12/03/2019, Day Shift 16:00–00:00 (08:00-16:00) - UTC (PT)
State of H1: Locked
Intent Bit: Observing
Weather: 0-5 mph wind
Primary 0.03 – 0.1Hz: 0.01 um/s
Secondary 0.1 – 0.3Hz: 0.6 um/s
Outgoing Operator: Jeff
Quick Summary: Locked and Observing 24 hours, TJ is taking IFO down for GRD tests
Ran the STS Centering and T240 Centering check scripts. STS: 2019-12-03 05:18:06.357834 There are 1 STS proof masses out of range ( > 2.0 [V] )! STS B DOF X/U = 4.702 [V] All other proof masses are within range ( < 2.0 [V] ): STS A DOF X/U = -0.757 [V] STS A DOF Y/V = -1.084 [V] STS A DOF Z/W = -0.264 [V] STS B DOF Y/V = -0.495 [V] STS B DOF Z/W = -0.345 [V] STS C DOF X/U = 0.344 [V] STS C DOF Y/V = 0.882 [V] STS C DOF Z/W = -0.11 [V] STS EX DOF X/U = -0.176 [V] STS EX DOF Y/V = 0.231 [V] STS EX DOF Z/W = 0.265 [V] STS EY DOF X/U = 0.397 [V] STS EY DOF Y/V = -0.326 [V] STS EY DOF Z/W = 0.693 [V] Assessment complete. T240: 2019-12-03 05:18:30.363749 There are 8 T240 proof masses out of range ( > 0.3 [V] )! ETMX T240 2 DOF X/U = -0.939 [V] ETMX T240 2 DOF Y/V = -1.005 [V] ETMX T240 2 DOF Z/W = -0.575 [V] ITMX T240 1 DOF X/U = -1.212 [V] ITMX T240 3 DOF X/U = -1.227 [V] ITMY T240 2 DOF Y/V = 0.3 [V] ITMY T240 3 DOF X/U = -0.43 [V] ITMY T240 3 DOF Z/W = -1.541 [V] All other proof masses are within range ( < 0.3 [V] ): ETMX T240 1 DOF X/U = 0.03 [V] ETMX T240 1 DOF Y/V = -0.038 [V] ETMX T240 1 DOF Z/W = -0.111 [V] ETMX T240 3 DOF X/U = -0.101 [V] ETMX T240 3 DOF Y/V = -0.058 [V] ETMX T240 3 DOF Z/W = -0.05 [V] ETMY T240 1 DOF X/U = -0.116 [V] ETMY T240 1 DOF Y/V = 0.251 [V] ETMY T240 1 DOF Z/W = -0.048 [V] ETMY T240 2 DOF X/U = -0.029 [V] ETMY T240 2 DOF Y/V = -0.086 [V] ETMY T240 2 DOF Z/W = -0.08 [V] ETMY T240 3 DOF X/U = -0.061 [V] ETMY T240 3 DOF Y/V = -0.118 [V] ETMY T240 3 DOF Z/W = 0.115 [V] ITMX T240 1 DOF Y/V = 0.146 [V] ITMX T240 1 DOF Z/W = 0.153 [V] ITMX T240 2 DOF X/U = 0.132 [V] ITMX T240 2 DOF Y/V = 0.214 [V] ITMX T240 2 DOF Z/W = 0.219 [V] ITMX T240 3 DOF Y/V = 0.111 [V] ITMX T240 3 DOF Z/W = -0.012 [V] ITMY T240 1 DOF X/U = 0.17 [V] ITMY T240 1 DOF Y/V = 0.145 [V] ITMY T240 1 DOF Z/W = 0.22 [V] ITMY T240 2 DOF X/U = 0.107 [V] ITMY T240 2 DOF Z/W = 0.236 [V] ITMY T240 3 DOF Y/V = 0.209 [V] BS T240 1 DOF X/U = 0.014 [V] BS T240 1 DOF Y/V = -0.093 [V] BS T240 1 DOF Z/W = 0.298 [V] BS T240 2 DOF X/U = 0.095 [V] BS T240 2 DOF Y/V = 0.271 [V] BS T240 2 DOF Z/W = 0.013 [V] BS T240 3 DOF X/U = 0.09 [V] BS T240 3 DOF Y/V = -0.149 [V] BS T240 3 DOF Z/W = -0.162 [V] Assessment complete. Closing FAMIS #9248
Reasonable first half of the shift. The IFO remained locked through a few earthquakes, and a Squeezer dropout. We were observing all but about half an hour while sorting out the Squeezer issue. Microseism remains elevated, but is not causing problems. Range has been floating around 115Mpc. Had one SubGRB alert, #E356318. Ignored due to high latency (12,824 seconds).
At 09:42 (01:42) drop out of Observing due to Squeezer. SQZ_MANAGER was stalled at LOCK_LO. The SQZ_LO_LR was stalled on LOCKING_OMC. There were three SDF Diffs reported. First I reverted the SDF diffs in the attached file, which did not resolve the issue. At 10:11 (02:11) I took the SQZ_MANAGER to NO_SQUEEZING. When the SQZ_MANAGER was happy at NO_SQUEEZING, selected SQUEEZING. The SQZ_LO_LR node relocked the OMC and the SQZ_MANAGER node arrived at the SQUEEZING state. With all involved nodes back to green, at 10:11 (02:11) put the IFO back into Observing mode.
TITLE: 12/02 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC STATE of H1: Observing at 114Mpc INCOMING OPERATOR: Jeff SHIFT SUMMARY: Remained locked and in observing. No issues. LOG: 00:47 UTC Set INJ_TRANS back to INJECT_SUCCESS 06:58 UTC GRB-Short E356311, INJ_TRANS set to INJECT_KILL, confirmed with LLO Fermi 597049044 TRIGGER_DUR: 0.512 [sec] Stand Down 07:48 UTC Set INJ_TRANS back to INJECT_SUCCESS
Have remained locked and in observing. No issues.
Laser Status:
Front End Power is 32.33W (should be around 30 W)
70W Output Power is 69.29W
Front End Watch is GREEN
70W Watch is GREEN
PMC:
It has been locked 27 days, 6 hr 55 minutes (should be days/weeks)
Reflected power = 11.59Watts
Transmitted power = 52.38Watts
PowerSum = 63.97Watts.
FSS:
It has been locked for 0 days 10 hr and 55 min (should be days/weeks)
TPD[V] = 2.746V (min 0.9V)
ISS:
The diffracted power is around 2.5%
Last saturation event was 0 days 10 hours and 55 minutes ago (should be days/weeks)
Possible Issues:
TPD[V] = 2.746V (min 0.9V) - Jason will be addressing this during Maintenance tomorrow
J. Kissel I've processed the ETMX longitudinal actuation strength data from 2019-11-11 (see day-of LHO aLOG 53162) for each of the three stages. In doing so, I've trended the front-end produced time-dependent correction factors to see if the systematic error between "the actuator strength used in the currently-still-in-use 2019-09-09 model" (which is actually the same numbers we've been using since the start of O3 in April 2019) and "the fit to the 2019-11-11 data" is small (i.e. the percent difference is unity). In short -- it's the perennial question: "anything that shows evidence that we need to update the model?" or, asked differently, "any evidence that the time dependent correction factors are not doing their job?" Summary: All continues to be well-understood and accurate at the level of concern needed with the O3 ETMX actuator. But, beneath that level of concern, there are interesting things happening. The attached .pdfs are the plots that show the fits to the 2019-11-11 data. Below is a table highlighting the results which help answer the above question: UIM PUM TST Reference Model 7.67e-08 (N/ct) 6.036e-10 (N/ct) 4.727e-12 (N/ct) MCMC Fit 8.038e-08 6.086e-10 4.780e-12 kappa via MCMC 1.0478 1.00824 1.01124 kappa via CALCS 0.997 1.013 1.012 Systematic Error 0.048% -0.0047% 0.00075% (MCMC-CALCS)/MCMC I was at first a bit alarmed by the 4% systematic error in the UIM, but the MCMC fit appears to be poor -- spoiled by the data point at 10 Hz -- so I've resumed a state of zen. Also, we've shown that a 5% scale factor systematic error on the UIM has no significant impact on the overall response function systematic error above 10 Hz, so even if the systematic error was real, I'd still brush it off. The previous time this table was made was for the 2019-10-28 measurement. See LHO aLOG 52973. Then, all actuation fits agreed with TDCFs within 0.5%, so I argue that the TDCFs are tracking reality well. Finally, the image attachment is a 1 month trend of the time-dependent correction factors, just because they turned out to be interesting -- but not consequential -- when I looked. The interesting points: (1) The high duty cycle from this month allows us to *clearly* resolve a six-hour period, 0.2% amplitude fluctuation in the ESD actuation strength. I attribute this to the first time LHO has demonstrably seen the impact of differential reaction-chain to main-chain distances changing due to tidal control. This change in chain-to-chain distance means the gap between reaction mass (the ESD pattern) and the test mass changes, thus impacting the ESD actuation strength. LLO regularly sees this because they drive all of their tidal control to *only* the main chain at the TOP mass level, as opposed to LHO which drives tidal to the UIM stage. This actuation strength fluctuation is real, noticeable, but at the 0.2% level, it doesn't really matter, especially since we're correcting for it. (2) The actuation strength as drifted down in 1 month, from 1.018 to its current value of 1.008. Again, inconsequential, because we correct for it, but I wonder what effect is causing this slow drift. I can think of three things that would evolve on that time scale: vacuum pressure, residual charge, and ambient temperature. The former two won't have changed over the course of the month, but trending further back in time reveals that that was a ~1% increase in drive strength between the end of O3A and the start of O3B, and the strength appears to just be returning to that level. The ambient temperature has changed for the colder over this month... (3) There has been significant step function in the PUM *and* UIM actuation strength, at the ~1% level after the 2019-11-26 the maintenance day break. The only activity that day at the end stations was powering up and leaving on the NCAL, setting up a PEM instrumentation array around it, and Robert making scattered light investigations. I could believe that the former two might have some electrical effect in the strength... but it's shear speculation. I'll be gathering a new set of data this Wednesday to continue the assessment, and confirm that we see this ~1% increase in the sweeps as well. I suspect it will.
In the above entry, I confusingly say
I've trended the [... TDCFs ...] to see if systematic error between [ ... the model of the actuation strength ...] and [the fit to this new data] is small (i.e. the percent difference is unity).
I meant either
- "the percent difference is close to zero" where percent difference is 100*[({new data fit} - model) / model] or
- "the ratio is unity," where the ratio is {new data fit} / model
and in my haste convolved the two.
And then, I totally botched the reporting of the percent difference, adding the *label* for %, without actually multiplying by 100.
For example, for the TST stage TDCF, the percent difference is
100 * (1.01124 - 1.012) / 1.012 = -0.075, and thus "0.075%"
and the ratio is just
1.01124 / 1.012 = 0.9992
Sorry for the confusion! It's too late for me to edit the entry proper, so please forgive the error and interpret with the corresponding grain of salt.
TITLE: 12/02 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 115Mpc
OUTGOING OPERATOR: Cheryl
CURRENT ENVIRONMENT:
SEI_CONF state: WINDY
Wind: 5mph Gusts, 4mph 5min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.38 μm/s
QUICK SUMMARY: No issues.
FAMIS 11246 Trends attached. ITMX is out of range in pitch and close to out of range in yaw. ETMX is close to out of range in pitch. ITMY is out of range in yaw.
J. Kissel
I think it might be time to redefine this FAMIS task. The above trends are only showing the last few days, and
(a) any number of things can happen (including several cycles of intentionally aligned vs. misaligned optics), and
(b) the +/-10 urad "range limits are not anysort of physical, functionality limits of the oplevs, they're merely "if the optic started at centered when we first started caring, then we'd be alarmed if the optical lever continually zoomed off 'in to the weeds' by 10 urad, or if the alignment was varying over 'days' by 10s of urads.
As the trends above stand, you can't tell either of these worrisome things from these from them.
Admittedly, there are other things to check for that the above trend would show, like
(a) Has the SUM dropped to some abnormally low value, or
(b) Does the few day trend show that the optical lever is *not* tracking the intentional alignments and misalignments, or
(c) Has there been some sort of "sudden" days worth of VEA temperature excursion
but we'd want to see these problems immediately, vs. the "once a week" that this test is made.
I attach a ~1 month ndscope trend, and I see that all of the optical levers (in PIT and YAW) for the QUADs, then all the Triples -- even though they did not *start* as "centered," they haven't drifted more than a few urad since the start of O3B. Thus, I would prefer that we keep the long term alignment reference rather than resetting this zero. Mostly because in the *future* month, we'll see a drop to zero, and we have to take the extra time to figure out if there was a centering that occurred.
Anyways -- just hoping start the conversation, with the goal being *my* usual -- let's think about the regular tests we do, 'cause maybe they're no longer needed and we can stop. OR if we're doing them regularly, let's make sure the results that we post reflect the questions we really want so we can answer them without further inspection. Further, if there are new tools out there that may answer the question better, let's use them (in this case, ndscope templates instead of python script).
J. Kissel As per our studies on Monday, Keita advised that we engage at least some level of analog low-pass on the QUADs' UIM coil drivers (see LHO aLOG 53376, and subsequent comments.) After we lost lock from an obscene amount of wind, I've now switched the ETMs to be in state 2, with *one* low pass on. This is the state I studied in LHO aLOG 53486. For ETMY, it was as easy as requesting state 2 and accepting it in the SDF system. However, for ETMX, I found that the ALS_DIFF guardian was requesting state 1 after every lock loss. So, I've changed the hard-coded request (which lived in the DOWN state), to be to request state 2. And also accepted this in the SDF. It looks like someone has already requested that the ITMs are in state 4 (with all three z:p = 10:1 filters ON), and accepted as such in SDF. These have been ON throughout the last ~24 hour lock stretch that started at the end of maintenance yesterday. State 4 should be fine as far as actuator saturations because we never send any drive request to the UIM stage of the ITMs. As such, turning on ALL low passes will only filter the DAC noise which is what we want.
Here's a screenshot of the UIM DAC requested drive during the first successful lock acquisition after switching on these low passes. I also attach an ASD of the UIM drive request during the noisiest part of the acquisition (when the DARM error signal is AS45Q.) We need more statistics regarding the early parts of the sequence (i.e. when ALS DIFF is acquiring) when the environment isn't awful [we had lots of sudden lock losses during the FIND_IR stages of ALS that couldn't be directly attributed to lack of range, but it's still suspicious], but it looks like we come quite close the DAC range when we're in the CARM reduction step, when DARM is on AS45Q. More data to come!
I have reverted ETMX's L1 back to state 1. We suspect that this (it having been set to 2) is the reason we've been losing lock in the carm offset reduction sequence lately. Keita pointed out that ETMY needed to be in state 2, since it gets a lot of low frequency actuation but no deliberate high frequency drive. However since ETMX gets length locking signal at a broad range of frequencies, it doesn't have the same quantization noise problems. So, ETMY is left in the lower noise (has an analog lowpass) state 2, but ETMX is back in the higher range (no analog lowpass) state 1. This has been changed in the ALS_DIFF guardian, and Niko is keeping an eye on the channel to make sure it's not written back to 2 somewhere else.
This has been accepted in SDF, so operators should not see an SDF diff when we get to NomLowNoise.
For the brief, 6-day, time period that the UIM LP1 filter was on between 2019-11-27 and 2019-12-03 (naturally, in between the regular calibration measurements that would have helped us quantify it more easily), the poor compensation of this driver's response manifests in systematic error in the DARM calibration.
Making a *very* long story short,
(a) The systematic error is very small, (of course frequency dependent, but at most) 0.15% and and 0.09 deg, and
(b) Performing this study reminded us that the original data upon which the compensation was based (see LHO aLOG 46927) is quite flawed, and this chassis should be remeasured and the compensation updated.
Attached is a prediction of what the calibration (response function) systematic error from this configuration change was over those fateful 6 days.
For details on how this estimate was created, see PART I of G2000527.
The script used to re-fit the data and make all corresponding plots for G2000527 lives here:
/ligo/svncommon/CalSVN/aligocalibration/trunk/Common/Electronics/H1/Scripts/
fit_ETMX_UIM_driver_20190203_IIRrational_20200401.py