Using the data collected by the locklost website, I have created some histograms on the locklosses we've experienced durig O3, filtered for different lockloss states/additional factors. The plots are located on my public html, and have been created for three time spans before commissioning month started: one week before, one month before, and for the O3 run so far. They are separated as follows:
H1_duration_*.png: histogram of lock lengths where we were in Observe when lockloss occurred
H1_states_*.png: histogram of states from which we have lost lock
H1_*_sats_*.png: histogram of which suspensions saturated first, from our three most common lockloss states: LOCKING_ALS, ACQUIRE_DRMI_1F, and NLN (in Observing)
With a recent lockloss release/re-run on O3 measurements, these can be filtered further for high ground motion/wind and the other established lockloss tags that have been created. Attached are three plots for the O3 run.
The new tags are a really useful addition to the lockloss tool. Check them out here.
By clicking on this link I could within a few minutes search for all of the locklosses from nominal low noise (guardian state 600) from the start of O3 until now and see how many of them have various tags.
- Out of 262 locklosses from Nominal low noise, 176 of them were from OBSERVE (I find it suprising that there were so many not in OBSERVE).
- 39 of these nominal low noise locklosses (15%) were due to ADS excursions caused by glitches drowning out the ADS lines. This is a lockloss that we can prevent in the future, perhaps by reducing the ADS frequencies or just by limiting the ADS error signals.
- 55 locklosses were tagged as windy (21%) and 66 were tagged as seismic (25%). Some locklosses have multiple tags.
- 13 locklosses (5%) were tagged as having analog board saturations, mostly IMC SPLITMON. Most of these locklosses do not have environment tags, which is a bit suprising so this is another type of lockloss that we should be able to mitigate.
I'd also like to highlight that it is worth taking an extra look at some of the plots that Niko has added in his directory linked above. For example, but comparing the state from which we lost lock over the last month to the same plot for all of O3a, you can see that the locklosses that happen durring the ASC engagement states (430 and 435) became much more of a problem in the last month of the run, but there were fewer ALS locklosses (15 and 16). Focusing on the ASC locklosses will be helpfull because each of these costs us significant time.
Another interesting point that Sheila brought up that I'd like to explain a little better: "Why do did H1's O3A have almost a factor 2 more maintenance time than in O1 and O2?" She confirms her impression is from information on the time accounting tabs on the summary pages for O3A, O2 and O1. As always, we must take the accuracy of these percentages with a grain of salt, because they are human entered statuses, and the criteria for putting the observatory mode in one state vs. another depends on the human operator corps, and often the lines between categories are blurry. That caveat aside, looking into this a little deeper, recall that "Maintenance" is actually a concatenation of several categories: Calibration, Preventative, Maintenance, and Corrective maintenance. So here, I summarize the data for the three runs: O1 O2 O3 Commissioning 2.8 3.4 3.5 Maintenance 4.4 5.4 8.6 CALI 0.7 0.4 1.1 PREV 1.8 3.3 3.5 CORR 1.9 1.7 3.9 Planned Eng. 0.1 11.9 0.0 ^ Holiday break So the source of the extra maintenance is a factor of 2 more of both calibration time as well as corrective maintenance. The amount of time spent on corrective maintenance is unavoidable / uncontrollable, and the extra time spent on calibration is for two reasons: (1) ER14, the engineering run leading up to O3, was reduced from the 4 weeks (which we had prior to O1 and O2) to 2 weeks (which we sacrificed to spend more time commissioning), and (2) We had several new / confusing new problems with DARM loop that required "more investigation than usual:" (a) the systematic error in the ETMX PUM which we now know resulted from unwanted L2A2L coupling as a result of using IFO spot positions that were "far" away from the optics' geometric center of rotation (because we had point absorbers, which newly started grossly impacting the IFO's control system after we increased both the laser power and the power recycling gain) (b) As a result (we think) of off-center spot positions, and mode mis-match between the arms and the signal recycling cavity, we had compounding effects of a drastically detuned SRC response in the sensing function AND left over impacts of parasitic L2A2L coupling. Seems to all add up, and not actually that we've taken that much longer for "preventative maintenance" [what operators have been coached to put the observatory mode in during maintenance Tuesdays].