Displaying reports 13901-13920 of 84136.Go to page Start 692 693 694 695 696 697 698 699 700 End
Reports until 13:32, Monday 21 August 2023
H1 SUS (CDS, SUS, SYS)
jeffrey.kissel@LIGO.ORG - posted 13:32, Monday 21 August 2023 (72356)
HAMs 3 thru 7, BSC 123 SUS Computer Clock Cycle Usage Thus Far during O4 :: All User Model SUS that are to be on-the-move below Run Below 25% Usage
J. Kissel

As I continue game-planning the integration of O5 suspensions into the existing system, (see progress thus far in G2301306) one of the questions is the percentage of computer usage needed on a given computer core running different numbers and amounts of suspensions. Given the sheer amount of new SUS coming in coupled with the limited number of computers and IO chassis, we'll need to pull a similar computer core assignment re-arrangement game that we've done before previous observing runs -- (for O4 see G2101636, and implementation aLOGs LHO:59651 for HAM56 and LHO:59669 for HAM7. For O3 see LHO:38827).

Summary of findings shown in the attached trends of the CPU_METER_MAX values for SUSB123 and SUSH34 and SUSH56 and SUSH7 since May 24 2023 16:00 UTC: 

    :: All 65 kHz IOP Models, be it SUSH34, SUSB123, SUSH56, or SUSH7 are consistently using 8 usec / 15 usec ~ 53% of capacity.

    On SUSB123, 
    :: The ITM 16 kHz user models consume the most clock cycle time, running with maximums of ~26 usec / 61 usec ~42%. (I presume the abnormally large computation comes from violin mode damping)
    :: The BS 16 kHz user model only consumes 14 / 61 ~ 23% of its core.

    On SUSH34,
    :: All three HSTSs, MC2, PR2, and SR2 run between 13 / 61 usec ~ 21% of core capacity. MC2 had "run the longest" at 14 and I presume that's from the fast longitudinal global control filtering (where PR2 and SR2 only receive slow angular drift control), but the restart after reboot from the dolphin crash on July 30 2023 (see LHO:71829) seems to have brought it back in symmetry with the other two at 13.

    On SUSH56 and SUSH7,
    :: all SUSH56 and SUSH7 16 kHz user models are using at most only 15 / 61 usec ~ 25% (it's SUSSRM running one globally controlled HSTS, and SUSSQZIN running three ZMs [two HPDS and one HDDS] and the OPO). Some, like SUSSQZOUT which are only running damping loops on ZM4, ZM5, and ZM6 top masses (two HPDSs and one HSDS, respectively) are using as little as 8 / 61 ~13%
Their 65 kHz IOPs are running at ~50% capacity.
    :: The SUSAUXH7 model is running at 4 kHz, and consumes only 4 / 244 ~ 6% of its core on SUSH7. I'll note that no one has populated the coil driver monitor filter banks with any filters, but I'm sure even if it was chock full I doubt it would even come close to "breaking the bank."


This bodes well for re-organizing models for O5; we're not computationally limited in doing so for any suspensions on the move, even if we keep running things at 16 kHz. Further, we can consider combining even more suspensions' controls into single models if we expect the SUS's filter computation to be light [e.g. only damping like in SUSSQZOUT].
Images attached to this report
LHO General
austin.jennings@LIGO.ORG - posted 12:59, Monday 21 August 2023 (72355)
Mid Shift Report

H1 has been locked for 20 hours, all systems appear stable and seismic motion is low.

LHO VE
david.barker@LIGO.ORG - posted 11:02, Monday 21 August 2023 (72353)
Mon CP1 Fill

Mon Aug 21 10:08:31 2023 INFO: Fill completed in 8min 27secs

Travis confirmed a good fill curbside.

Images attached to this report
LHO General
austin.jennings@LIGO.ORG - posted 08:00, Monday 21 August 2023 (72348)
Ops Day Shift Start

TITLE: 08/21 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 151Mpc
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 3mph Gusts, 2mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.18 μm/s 
QUICK SUMMARY:

- H1 locked for 15.5 hours

- SEI/CDS ok

LHO General
ryan.short@LIGO.ORG - posted 00:00, Monday 21 August 2023 (72346)
Ops Eve Shift Summary

TITLE: 08/20 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 151Mpc
INCOMING OPERATOR: Corey
SHIFT SUMMARY: Quiet shift; recovered from last shift's EQ lockloss and kept observing throughout. H1 has now been locked for 7.5 hours.

LOG:

No log for this shift.

LHO General
ryan.short@LIGO.ORG - posted 20:00, Sunday 20 August 2023 (72347)
Ops Eve Mid Shift Report

State of H1: Observing at 153Mpc

H1 was able to relock automatically following the lockloss from the last shift and has now been observing for 3.5 hours.

LHO General
ryan.short@LIGO.ORG - posted 16:02, Sunday 20 August 2023 (72345)
Ops Eve Shift Start

TITLE: 08/20 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Austin
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 6mph Gusts, 4mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.19 μm/s
QUICK SUMMARY: H1 is relocking following an earthquake-caused lockloss, currently up to ENGAGE_ASC_FOR_FULL_IFO

LHO General
austin.jennings@LIGO.ORG - posted 16:00, Sunday 20 August 2023 (72341)
Sunday Operator Summary

TITLE: 08/20 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Lock Aquisition
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY:

- 20:19 - inc 5.7 EQ from South Sandwich Islands

- Superevent S230820bq - call received at 21:26 UTC, no alert on verbal

- 21:43 - inc 5.0 EQ from SoCal

- H1 did an auto initial alignment, just completed and now relocking as I leave the IFO to Ryan S.

LOG:

No log for this shift.

H1 General (Lockloss)
austin.jennings@LIGO.ORG - posted 14:50, Sunday 20 August 2023 (72344)
Lockloss @ 21:47

Lockloss @ 21:47, caused by local EQs.

LHO General
austin.jennings@LIGO.ORG - posted 13:21, Sunday 20 August 2023 (72343)
Mid Shift Report

H1 has been locked for 45 hours. Nothing to note locally, but we just got an alert for a 5.7 EQ from South Sandwich Islands region.

LHO VE
david.barker@LIGO.ORG - posted 10:16, Sunday 20 August 2023 (72342)
Sun CP1 Fill

Sun Aug 20 10:06:46 2023 INFO: Fill completed in 6min 42secs

Images attached to this report
LHO General
austin.jennings@LIGO.ORG - posted 08:00, Sunday 20 August 2023 (72340)
Ops Day Shift Start

TITLE: 08/20 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 149Mpc
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 7mph Gusts, 4mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.16 μm/s 
QUICK SUMMARY:

- H1 still going strong, right now at a 39:45 hour lock

- Systems stable, ground motion low

- Optics lab dust monitor alarm on alert handler - possibly wild fires?

LHO General
ryan.short@LIGO.ORG - posted 00:00, Sunday 20 August 2023 (72338)
Ops Eve Shift Summary

TITLE: 08/19 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 150Mpc
INCOMING OPERATOR: Corey
SHIFT SUMMARY:

H1 has now been locked for 32 hours. Very quiet evening with observation time throughout.

The EY VEA dust monitor alarm started up again shortly after mid-shift. Attached is a trend of the dust counts over the past 8 hours.

LOG:

No log for this shift.

Images attached to this report
LHO General
ryan.short@LIGO.ORG - posted 20:00, Saturday 19 August 2023 (72339)
Ops Eve Mid Shift Report

State of H1: Observing at 150Mpc

H1 has now been locked for 28 hours. Quiet evening on site; the EY VEA dust monitor alarm has stopped.

LHO General
ryan.short@LIGO.ORG - posted 16:10, Saturday 19 August 2023 (72337)
Ops Eve Shift Start

TITLE: 08/19 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 152Mpc
OUTGOING OPERATOR: Austin
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 17mph Gusts, 15mph 5min avg
    Primary useism: 0.05 μm/s
    Secondary useism: 0.16 μm/s
QUICK SUMMARY: H1 has now been locked for 24 hours.

LHO General
austin.jennings@LIGO.ORG - posted 16:00, Saturday 19 August 2023 (72334)
Saturday Operator Summary

TITLE: 08/19 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 152Mpc
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY:

- EY DM 1 (H1:PEM-EY_DUST_VEA1_300NM_PCF) has been intermittently going off on alert handler - possibly due to wildfires, scope attatched

- Superevent S230819ax - call received at 17:21 UTC, no alert on verbal

- Overall a very quiet (and smoky) day, leaving the IFO to Ryan S. just under a 24 hour lock
LOG:

No log for this shift.

Images attached to this report
H1 PEM (DetChar)
robert.schofield@LIGO.ORG - posted 19:22, Friday 18 August 2023 - last comment - 11:13, Monday 11 September 2023(72331)
DARM 52 Hz peak from chilled water pump at EX: HVAC shutdown times

Genevieve, Lance, Robert

To further understand the roughly 10Mpc lost to the HVAC (https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=72308), we made several focussed shutdowns today. These manipulations were made during observing (with times recorded) because such HVAC changes happen automatically during observing and also, we were reducing noise rather than increasing it. The times of these manipulations are given below.

One early outcome is that the peak at 52 Hz in DARM is produced by the chilled water pump at EX (see figure). We went out and looked to see if the vibration isolation was shorted, it was not, though there are design flaws (the water pipes arent isolated). We switched from CHWP-2 to CHWP-1 to see if the particular pump was extra noisy. CHWP-1 produced a similar peak in DARM at its own frequency. The peak in accelerometers is also similar in amplitude to the one from the water pump at EY. One possibility is that the coupling at EX is greater because of the undamped cryobaffle at EX.

 

Friday HVAC shutdowns; all times Aug. 18 UTC

15:26 CS SF1, 2, 3, 4 off

15:30:30 CS SF5 and 6 off

15:36 CS SF5 and 6 on

15:40 CS SF1, 2, 3, 4 back on

 

16:02 EY AH2 (only fan on) shut down

16:10 EY AH2 on

16:20 EY AH2 off

16:28 EY AH2 on

16:45 EY AH2 and chiller off

16:56:30 EY AH2 and chiller on

 

17:19:30 EX chiller only off, pump stays on

17:27 EXwater pump CHWP-2 goes off

17:32: EX CHWP-2 back on chiller back on right after

 

19:34:38 EX chiller off, CHWP-2 pump stays on for a while

19:45 EX chiller back on

 

20:20 EX started switch from chiller 2 to chiller 1 - slow going

21:00 EX Finally switched

21:03 EX Switched back to original, chiller 1 to chiller 2

 

Non-image files attached to this report
Comments related to this report
jeffrey.kissel@LIGO.ORG - 10:29, Monday 21 August 2023 (72350)DetChar, FMP, ISC, OpsInfo
Turning Roberts reference to LHO:72308 into a hyperlink for ease of navigation.

Check out LHO:72297 for a bigger picture representation of how the 52 Hz peak in the broader DARM sensitivity, and from the Time Stamps in Elenna's plots, they were taken at 15:27 UTC, just after the corner station (CS) "SFs 1, 2, 3, 4" are off.

SF stands for "Supply Fans" i.e. those air handler unit (AHU) fans that push the cool air in to the LVEA. Recall, there are two fans per air handler unit, for the two air handler units (AHU1 and AHU2) that feed the LVEA in the corner station.

The channels that you can use to track the corner station's LVEA HVAC system are outlined more in LHO:70284, but in short, you can check the status of the supply fans via the channels
    H0:FMC-CS_LVA_AH_AIRFLOW_1   Supply Fan (SF) 1
    H0:FMC-CS_LVA_AH_AIRFLOW_2   Supply Fan (SF) 2
    H0:FMC-CS_LVA_AH_AIRFLOW_3   Supply Fan (SF) 3
    H0:FMC-CS_LVA_AH_AIRFLOW_4   Supply Fan (SF) 4
jeffrey.kissel@LIGO.ORG - 13:13, Monday 28 August 2023 (72486)DetChar, ISC, SYS
My Bad: -- Elenna's aLOG is showing the sensitivity on 2023-Aug-17, and the Robert logging of times listed above are for 2023-Aug-18. 

Elenna's demonstration is during Robert's site-wide HVAC shut down 2023-Aug-17 -- see LHO:72308 (i.e. not just the corner as I'd errantly claimed above.).
jeffrey.kissel@LIGO.ORG - 11:13, Monday 11 September 2023 (72805)FMP, ISC, OpsInfo
For these 2023-Aug-18 times mentioned in this LHO aLOG 72331, check out the subsequent analysis of impact in LHO:72778.
H1 PEM (DetChar)
robert.schofield@LIGO.ORG - posted 17:53, Thursday 17 August 2023 - last comment - 10:26, Monday 21 August 2023(72308)
PEM activites Wed. and Thurs.: PEM injections, bias sweep, HVAC shutdown breaks 160 Mpc (for a few minutes)

Lance, Genevieve, Robert

PEM injections

We are redoing part of the formal PEM injection program, which took place for a week before the run, because the 75-60W change reduced vibration coupling by factors of 2 or 3. On Wednesday, during the commissioning slot, we finished acoustic injections at EY and started shaker injections at EY. Thursday we finished the main shaker injections at EY.

ETMY bias sweep with electronic ground injections

Fig. 1 shows that the ETMY bias setting that minimizes our electronic ground injection coupling has possibly changed a little, but not much, since January.

HVAC shutdown

I shut down the HVAC, site-wide, from about 15:23 to 15:37 Aug 17 UTC. The range increased by a little less than 10 Mpc to a little over 160 Mpc. Elenna posted a spectrum for the best part of the site-wide shutdown (see below). The shutdown and especially the re-start took time, so Fig. 2 shows just the effect of of the chilled water system at EX. Also, we shut down SF4 only (the fan pushing 11,000 CFM at the CS, from 21:18 to 21:32 UTC, but we did not see an improvement in range. We will continue with a focussed study tommorrow in order to better understand the locations of the problems.

Non-image files attached to this report
Comments related to this report
jeffrey.kissel@LIGO.ORG - 10:26, Monday 21 August 2023 (72349)DetChar, FMP, ISC
For the ASD of GDS-CALIB_STRAIN of Elenna's that Robert mentions comparing the HVAC ON vs. OFF times see LHO:72297.
H1 ISC (PEM)
elenna.capote@LIGO.ORG - posted 11:38, Thursday 17 August 2023 - last comment - 13:15, Monday 28 August 2023(72297)
DARM with and without HVAC

Robert did an HVAC off test. Here is a comparison of GDS CALIB STRAIN NOLINES from earlier on in this lock and during the test. I picked both times off the range plot from a time with no glitches.

Improvement from removal of 120 Hz jitter peak, apparent reduction of 52 Hz peak, and broadband noise reduction at low frequency (scatter noise?).

I have attached a second plot showing the low frequency (1-10 Hz) spectrum of OMC DCPD SUM, showing no appreciable change in the low frequency portion of DARM from this test.

Images attached to this report
Comments related to this report
jeffrey.kissel@LIGO.ORG - 14:57, Thursday 17 August 2023 (72302)DetChar, FMP, OpsInfo, PEM
Reminders from the summary pages as to why we got so much BNS range improvement from removing the 52 Hz and 120 Hz features shown in Elenna's ASD comparison.
Pulled from https://ldas-jobs.ligo-wa.caltech.edu/~detchar/summary/day/20230817/lock/range/.

Range integrand shows ~15 and ~5 MPC/rtHz reduction at the 52 and 120 Hz features.

BNS range time series shows brief ~15 MPC improvement at 15:30 UTC during Robert's HVAC OFF tests.
Images attached to this comment
elenna.capote@LIGO.ORG - 11:50, Friday 18 August 2023 (72321)

Here is a spectrum of the MICH, PRCL, and SRCL error signals at the time of this test. The most visible change is the reduction of the 120 Hz jitter peak also seen in DARM. There might be some reduction in noisy peaks around 10-40 Hz in the signals, but the effect is small enough it would be useful to repeat this test to see if we can trust that improvement.

Note: the spectra have strange shapes, I think related to some whitening or calibration effect that I haven't bothered to think about to make these plots. I know we have properly calibrated versions of the LSC spectra somewhere, but I am not sure where. For now these serve as a relative comparison.

Images attached to this comment
jeffrey.kissel@LIGO.ORG - 10:46, Monday 21 August 2023 (72352)DetChar, FMP, PEM
According to Robert's follow-up / debrief aLOG (LHO:72331) and the time-stamps in the bottom left corner of Elenna's DTT plots, she's is using the time 2023-08-17 15:27 UTC, and that corresponds to the time when Robert had turned off all four the supply fans (SF1, SF2, SF3, and SF4) in the corner station (CS) air handler units (AHU) 1 and 2 that supply the LVEA around 2023-08-17 15:26 UTC.
jeffrey.kissel@LIGO.ORG - 13:15, Monday 28 August 2023 (72487)DetChar, PEM, SYS
My Bad: -- Elenna's aLOG is showing the sensitivity on 2023-Aug-17, and the Robert LHO:72331 logging of times listed above are for 2023-Aug-18. 

Elenna's demonstration is during Robert's site-wide HVAC shut down 2023-Aug-17 -- see LHO:72308 (i.e. not just the corner as I'd errantly claimed above.).
H1 SUS (DetChar, INJ, ISC, OpsInfo)
jeffrey.kissel@LIGO.ORG - posted 09:52, Tuesday 15 August 2023 - last comment - 14:26, Monday 23 October 2023(72221)
ETMX M0 Longitudinal Damping has been fed to TMTS M1 Unfiltered Since Sep 28 2021; Now OFF.
J. Kissel, J. Driggers

I was brainstorming why LOWNOISE_LENGTH_CONTROL would be ringing up a Transmon M1 to M2 wire violin mode (modeled to be at 104.2 Hz for a "production" TMTS; see table 3.11 of T1300876) for the first time on Aug 4 2023 (see current investigation recapped in LHO:72214), and I remembered "TMS tracking..."

In short: we found that ETMX M0 L OSEM damping error signal has been fed directly to TMSX M1 L path global control path, without filtering, since Sep 28 2021. Yuck!

On Aug 30 2021, I resolved the discrepancies between L1 and H1 end-station SUS front-end models -- see LHO:59772. Included in that work, I cleaned up the Tidal path, cleaned up the "R0 tracking" path (where QUAD L2 gets fed to QUAD R0), and installed the "TMS tracking" path as per ECR E2000186 / LLO:53224. In short, "TMS tracking" couples the ETM M0 longitudinal OSEM error signal to the TMS M1 longitudinal "input to the drivealign bank" global control path, with the intent of matching the velocity of the two top masses to reduce scattered light.

On Aug 31 2021, the model changes were installed during an upgrade to the RCG -- see LHO:59797, and we've confirmed that I turned both TMSX and TMSY paths OFF, "to be commissioned later, when we have an IFO, if we need it" at
    Tuesday -- Aug 31 2021 21:22 UTC (14:22 PDT) 

However, 28 days later,
    Tuesday -- Sept 28 2021 22:16 UTC (15:16 PDT)
the TMSX filter bank got turned back on, and must have been blindly SDF saved as such -- with no filter in place -- after an EX IO chassis upgrade -- see LHO:60058. At the time, that RCG 4.2.0 still had the infamous "turn on a new filter with its input ON, output ON, and a gain of 1.0" feature, that has been since resolved with RCG 5.1.1. So ... maybe, somehow, even though the filter was already installed on Aug 31 2021, the IO chassis upgrade rebuild, reinstall, and restart of the h1sustmsx.mdl front end model re-registered the filter as new? Unclear. Regardless this direct ETMX M0 L to TMSX M1 L path has been on, without filtering, since Sep 28 2021. Yuck!

Jenne confirms the early 2021 timeline in the first attachment here.
She also confirms via a ~2 year trend of the H1:SUS-TMSY_M1_FF_L filter bank's SWSTAT, that no filter module has *ever* been turned on, confirmed that there's *never* been filtering.

Whether this *is* the source of 102.1288 Hz problems and that that frequency is the TMSX transmon violin mode is still unclear. Brief investigations thus far include
    - Jenne briefly gathered ASDs of ETMX M0 L (H1:SUS-ETMX_M0_DAMP_L_IN_DQ) and TMSX M1 L OSEMs' error signal (H1:SUS-TMSX_M1_DAMP_L_IN1_DQ) around the time of Oli's LOWNOISE_LENGTH_CONTROL time, but found that at 100 Hz, the OSEMs are limited by their own sensor noise and don't see anything.
    - She also looked through the MASTER_OUT DAC requests (), in hopes that the requested control signal would show something more or different, but found nothing suspicious around 100 Hz there either.
    - We HAVE NOT, but could look at H1:SUS-TMSX_M1_DRIVEALIGN_L_OUT_DQ since this FF control filter should be the only control signal going through that path. I'll post a comment with this.

Regardless, having this path on with no filter is clearly wrong, so we've turned off the input, output, and gain accepted the filter as OFF, OFF, and OFF in the SDF system (for TMSX, the safe.snap is the same as the observe.snap).
Images attached to this report
Comments related to this report
jeffrey.kissel@LIGO.ORG - 10:39, Tuesday 15 August 2023 (72226)
No obvious blast in the (errant) path between ETMX M0 L and TMSX M1 L, the control channel H1:SUS-TMSX_M1_DRIVEALIGN_L_OUT_DQ, during the turn on of the LSC FF.

Attached is a screenshot highlighting one recent lock acquisition, after the addition / separation / clean up of calibration line turns ons (LHO:72205):
    - H1:GRD-ISC_LOCK_STATE_N -- the state number of the main lock acquisition guardian,
    - H1:LSC-SRCLFF1_GAIN, H1:LSC-PRCLFF_GAIN, H1:MICHFF_GAIN -- EPICs records showing the timing of when the LSC feed forward is turned on
    - The raw ETMX M0 L damping signal, H1:SUS-ETMX_M0_DAMP_L_IN1_DQ -- stored at 256 Hz
    - The same signal, mapped (errantly) as a control signal to TMSX M1 L -- also stored at 256 Hz
    - The TMSX M1 L OSEMs H1:SUS-TMSX_M1_DAMP_L_IN1_DQ, which are too limited by their own self noise to see any of this action -- but also only stored at 256 Hz.

In the middle of the TRANSITION_FROM_ETMX (state 557), DARM control is switching from ETMX to some other collection of DARM actuators. That's when you see the ETMX M0 L (and equivalent TMSX_M1_DRIVEALIGN) channels go from relatively noisy to quiet.

Then, at the very end of the state, or the start of the next state, LOW_NOISE_ETMX_ESD (state 558), DARM control returns to ETMX, and the main chain top mass, ETMX M0 gets noisy again. 

Then, several seconds later, in LOWNOISE_LENGTH_CONTROL (state 560), the LSC feed forward gets turned on. 

So, while there is control request changes to the TMS, at least according to channels stored at 256 Hz, we don't see any obvious kicks / impulses to the TMS during this transition. 
This decreases my confidence that something was kicking up a TMS violin mode, but not substantially.
Images attached to this comment
jeffrey.kissel@LIGO.ORG - 10:33, Wednesday 16 August 2023 (72275)DetChar, DetChar-Request
@DetChar -- 
This errant TMS tracking has been on throughout O4 until yesterday.

The last substantial nominal low noise segment before the this (with errant, bad TMS tracking) was on  
     2023-08-15       04:41:02 to 15:30:32 UTC
                      1376109680 - 1376148650
the first substantial nominal low noise segment after this change 
     2023-08-16       05:26:08 - present
                      1376198786 - 1376238848 

Apologies for the typo in the main aLOG above, but *the* channels to understand the state of the filter bank that's been turned off are 
    H1:SUS-TMSX_M1_FF_L_SWSTAT
    H1:SUS-TMSX_M1_FF_L_GAIN

if you want to use that for an automated way of determining whether the TMS tracking is on vs. off.

If the SWSTAT channel has a value of 37888 and the GAIN channel has a gain of 1.0, then the errant connection between ETMX M0 L and TMSX M1 L was ON. That channels has now a value of 32768 and 0.0, respectively, indicating that it's OFF. (Remember, for a standard filter module a SWSTAT value of 37888 is a bitword representation for "Input, Output, and Decimation switches ON." A SWSTAT value of 32768 is the same bitword representation for just "Decimation ON.")

Over the next few weeks, can you build up an assessment of how the IFO has performed a few weeks before vs. few weeks after?
     I'm thinking, in particular, in the corner of scattered light arches and glitch rates (also from scattered light), but I would happily entertain any other metric you think are interesting given the context.

     The major difference being that TMSX is no longer "following" ETMX, so there's a *change* in the relative velocity between the chains. No claim yet that this is a *better* change or worse, but there's definitely a change. As you know, the creation of this scattered-light-impacting, relative velocity between the ETM and TMS is related to the low frequency seismic input motion to the chamber, specifically between the 0.05 to 5 Hz region. *That* seismic input evolves and is non-stationary over the few weeks time scale (wind, earthquakes, microseism, etc.), so I'm guessing that you'll need that much "after" data to make a fair comparison against the "before" data. Looking at the channels called out in the lower bit of the aLOG I'm sure will be a helpful part of the investigation.

I chose "a few weeks" simply because the IFO configuration has otherwise been pretty stable "before" (e.g., we're in the "representative normal for O4" 60 W configuration rather than the early O4 75 W configuration), but I leave it to y'all's expertise and the data to figure out a fair comparison (maybe only one week, a few days, or even just the single "before" vs. "after" is enough to see a difference).
ansel.neunzert@LIGO.ORG - 14:31, Monday 21 August 2023 (72357)

detchar-request git issue for tracking purposes.

jane.glanzer@LIGO.ORG - 09:12, Thursday 05 October 2023 (73271)DetChar
Jane, Debasmita

We took a look at the Omicron and Gravity triggers before and after this tracking was turned off. The time segments chosen for this analysis were:

TMSX tracking on: 2023-07-29 19:00:00 UTC - 2023-08-15 15:30:00 UTC, ~277 hours observing time
TMSX tracking off: 2023-08-16 05:30:00 UTC - 2023-08-31 00:00:00 UTC, ~277 hours observing time

For the analysis, the Omicron parameters chosen were SNR > 7.5, and a frequency between 10 Hz and 1024 Hz. The Gravity Spy glitches included a confidence of > 90%. 

The first pdf contains glitch rate plots. In the first plot, we have the Omicron glitch rate comparison before and after the change. The second and third plots shows the comparison of the Omicron glitch rates before and after the change as a function of SNR and frequency. The fourth plot shows the Gravity Spy classifications of the glitches. What we can see from these plots is that when the errant tracking was on, the overall glitch rate was higher (~29 per hour when on, ~15 per hour when off). It was particularly high in the 7.5-50 SNR range and 10Hz - 50Hz range, which is typically where we observe scattering. The Gravity Spy plot shows that scattered light is the most common glitch type when the tracking is both on and off, but reduces after the tracking is off.

We also looked into see if these scattering glitches were coincidence in "H1:GDS-CALIB_STRAIN" and "H1:ASC-X_TR_A_NSUM_OUT_DQ", which is shown in the last pdf. From the few examples we looked at, there does seem to be some excess noise in the transmitted monitor channel when the tracking was on. If necessary, we can look into more examples of this. 
Non-image files attached to this comment
debasmita.nandi@LIGO.ORG - 14:26, Monday 23 October 2023 (73674)
Debasmita, Jane

We have plotted the ground motion trends in the following frequency bands and DOFs

1. Earthquake band (0.03 Hz--0.1 Hz) ground motion at ETMX-X, ETMX-Z and ETMX-X tilt-subtracted
2. Wind speed (0.03 Hz--0.1 Hz) at ETMX
3. Micro-seismic band (0.1 Hz--0.3 Hz) ground motion at ETMX-X

We have also calculated the mean and median of the ground motion trends for two weeks before and after the tracking was turned off. It seems that while motion in all the other bands remained almost same, the microseismic band ground motion (0.1-0.3 Hz) has increased significantly (from a mean value of 75.73 nm/s to 115.82 nm/s) when the TMS-X tracking was turned off. Still, it produced less scattering than before when the TMS-X tracking was on. 

The plots and the table are the attached here.
Non-image files attached to this comment
Displaying reports 13901-13920 of 84136.Go to page Start 692 693 694 695 696 697 698 699 700 End