DQSHIFT 20240429-20240505

Here is DQ Shift Report During this week :

• This week's cycle is around 73.82% and Sunday is the lowest during EQs happened
• Ground motion during this week relative high correspond to many EQs happened over the world and around US also
• There is misalignment of IFO laser but has been resolved.
• In HAM2 and HAM5 show many noises during the week, also strong violin modes occured from lower to high frequency
• H1:ASC-DHARD_Y_OUT_DQ, and H1:PEM-EX_VMON_ETMX_ESDPOWER48_DQ Channel always appeared during this week.
• Magnetic Noise and ESD Rack Shows High Noises (both in ETM X and Y)
• In several days, PSL Table also shows unstability, and Sunday 05/05 the humidity (chiller is quite high)
• In FScan , Saturday and Sunday has not show any plot, but on the Friday 05/03 shows a high line counts (>1000) and the largest among 1 month duration
• In Bokeh Plot, in lower frequencies 9around 80-150 Hz, some combs appeared in the same spot during this week)

TITLE: 05/06 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 151Mpc
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY:

IFO is in NLN and OBSERVING (6 hr 31 min lock). This is the longest lock in the last 24 hrs (4 brief locks in between).

I did not experience any of the issues that occured earlier in the week.

The squeezer phase angle had to be adjusted after 2-3 hrs (noticed the range going down, then noticed the SQZ not at its optimum) and was able to bring the range back up to what it was prior.
 

LOG:

None

IFO is in NLN and Observing as of 00:33 UTC (2 hr 35 min lock)

Nothing else of note

Anamaria, Sheila, Robert

We scanned the biases for three test masses to find the coupling minimum for currents that we injected onto the building ground. The optimal ETMY bias was 115 V in January of 2023, 170 V in Aug. (72308) and 200 V now.  With ITMX at 0V, the optimal for ITMY was 60 V in March of 2023 (68053) and -222 V now, The figure shows some of the bias scans.

Anamaria, Robert

We photographed, from a second angle, the beamspot of a stray beam that I noticed last visit (76969 - Fig. 3 ). The photos (Figure 1) confirm that there is indeed a bright stray beam hitting, at a grazing angle, the bellows and other parts of the spool piece between HAM3 and the IMC tube.

I also wanted to measure the power of the stray beam that had been producing a large 48 Hz peak in DARM during early O3, in order to improve my scattering coupling calculations. This is the beam that reflects off of the PR2 scraper baffle and hits the illuminator viewport on HAM3, which we mitigated by inserting a black-glass viewport cover (52184). We were surprised by how bright the beam is, reaching 20 mW on the power meter (see Figure 2), even though we could not fit the whole beam on the meter. We estimate that the beam may reach 60  mW.

Finally, we made movies as we swept the ITMY compensation plate, hoping to see some indication of where the ghost beams are hitting, but did not. We will have to wait until Alena gives us the likely position of the CP ghost beams based on the angles that Anamaria showed us how to measure.

Lockloss due to local EQ in Idaho (EQ mode activated but wwasn't enough it seems).

TITLE: 05/05 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 152Mpc
INCOMING OPERATOR: Ibrahim
SHIFT SUMMARY:
Overall summery is that it's been earthquaking all day.

Another Incoming 5.6 mag Earthquake
Took this time to do another Initial_Alignment while the earthquake passed.

Reached Observing for the 3rd time today at 22:36 UTC

LOG:
no log

TITLE: 05/05 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 149Mpc
OUTGOING OPERATOR: Tony
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 16mph Gusts, 12mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.11 μm/s
QUICK SUMMARY:

IFO is in NLN and OBSERVING (47 min lock)

TITLE: 05/05 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 149Mpc
OUTGOING OPERATOR: Ryan S
CURRENT ENVIRONMENT:
    SEI_ENV state: EARTHQUAKE
    Wind: 19mph Gusts, 17mph 5min avg
    Primary useism: 0.15 μm/s
    Secondary useism: 0.13 μm/s
QUICK SUMMARY:
First lock:
Ran Initial Alignment
Started Locking 15:28 UTC
Held in ENGAGE_DRMI_ASC for a 3-4 Minutes minutes after reading alog
https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=77622

Had no problems in DRMI Locking.
Had no problems in Transition_From_ETMX

Incoming 5.4 Mag Earthquake from Papua New Guinea
https://earthquake.usgs.gov/earthquakes/eventpage/us6000mwhz/executive

NLN Reached at 16:16 UTC
Observing reached at 16:19 UTC

Unknown lockloss 16:47 UTC https://ldas-jobs.ligo-wa.caltech.edu/~lockloss/index.cgi?event=1398962851
Screenshots attached to alog. 

Relocking:
Tried a few times to get above DRMI. Made it to Check_AS_SHUTTERS then Lockloss.
Trying another Initial_Alignment just incase it is an Alignment issue before I start exploring more exotic issue.

5.4 Mag Earthquake off the coast of Ecuador
https://earthquake.usgs.gov/earthquakes/eventpage/us6000mwij/executive

Stopped at DRMI_LOCKED_PREP_ASC, Changed H1:LSC-PRCL1_GAIN from 8 to 4.
then proceeded to Transition_DRMI_To_3F where I stopped again to change the gain back to 8.

6.2 Mag Earthquake from Indonesia
https://earthquake.usgs.gov/earthquakes/eventpage/us6000mwjd/executive
I think we May have Survived this!

But we have 2 more 4.9's coming in from Chile.... I guess we will find out.

Sun May 05 10:13:13 2024 INFO: Fill completed in 13min 9secs

The atomic clock timing reference signal, which is being monitored by the MSR timing comparator, jumped by 0.29 seconds 04:16:16 UTC (21:16 Sat 04may2024 PDT).

This is the cause of the "C" in the timing status on the CDS Overview. This is a reference only, we will investigate Monday.

TITLE: 05/05 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Ryan S
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 15mph Gusts, 11mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.10 μm/s
QUICK SUMMARY:
When I arrived H1 was LOCKING_ALS.
I let it try to lock itself twice. First it made it up to LOCKING_ALS before a lockloss. and on the second attempt it made it up to Engage_DRMI_ASC.
I then took ISC Lock to Initial Alignment.

EX had some ground moovement with might be coupled with the wind as it seems to be gusting. Though it looked strange because EY did not have the same apparent ground motion. We are expecting an R wave to hit from a 5.4 Mag Earthquake as well.

So Hopefully after the IA we should be goo to get locked again.

 

TITLE: 05/05 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 157 Mpc
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY:

IFO is in NLN and OBSERVING as of 05:57 UTC (1hr 17 min lock)

Much of the shift was spent troubleshooting locking issues in the interferometer. There were mainly two issues. (lockloss alog 77620 and alog 77615).

BS Glitch/PRCL Buzzing 

This issue was characterized by locklosses that occured during TURN_ON_BS_STAGE_2 and ENGAGE_DRMI_ASC. There would be a kick seen primarily in the ST1 T240 and L4C channels (Screenshots 1 and 2). Initially, this was thought to be a BS glitch. I first noted this issue in alog 77615 but as this alog states, we have been experiencing this for quite a while. During the DAY shift, Sheila and Tony found that in fact, this may be an issue with the PRCL gain being too high, causing a 100Hz buzzing (Screenshot 3). Relevant alogs include: alog 77574, alog 77573 (which detail Solution 1), and alog 77583 (Solution 2)

Temporary Solution 1:

Wait in ENGAGE_DRMI_ASC for ASC to converge more than guardian needs it to in order to mitigate the effects of possible BS kicks. From roughly 9-10 locklosses of this species, this worked around 6-7 times.

Temporary Solution 2:

Request and stay in ACQUIRE_DRMI_1F, change the PRCL1 gain from 8 to 4, continue to TRANSITION_DRMI_TO_3F and then change it back. This worked every time I tried this (3 times) and is the better solution.

Transition from ETMX

This issue was characterized by locklosses that occured either towards the very end of TRANSITION_FROM_ETMX or just as guardian reached LOWNOISE_ESD_ETMX and is part of an ongoing trend that is mostly detailed in alog 77359 where every time this state is run, there is a glitch in the 10-15 seconds leading upto the transtion of states. The alog notes that while investigations are ongoing, this seems to be something that ramp time and filter changes would fix. (screenshot 4 and 5). The alog also notes that locklosses in this state are most prevalent during periods of high wind. This issue has NOT been fixed and we were only able to survive it due to decreasing wind microseism. As we approached this state, after losing lock here twice, I checked the wiki/Jenne's troubleshooting doc and made calls to 3 people on the call list but either they could not help or there was no answer. Thankfully, a 4th person's Saturday night wasn't interrupted because the wind calmed down enough to survive this. Screenshot 4 shows a lockloss during this state while Screenshot 5 shows us surviving it - both have the glitch.

Other:

• Initial alignment ran fully automatically, which the lock acquisition survey shows did not happen recently. Initial alignment did not fix either issue.
• Find_IR was particularly slow, failing once and requiring DIFF intervention.
• ALS was particularly unstable after the second lockloss.
• DRMI was actually catching quite quickly (but not staying through to BS_2 due to aforementioned issue) - in 10 DRMI locks, we only went through PRMI twice and we did not go through MICH at all.
• Violins looked splendid the whole time (Thanks Rahul and Tony!)
• We survived a 5.6 Mag EQ

I expect to continue investigating both these issues tomorrow and wanted to get to them today but spent around 6.5 hours locking and 0.5 hrs eating. Given we are in observing for sufficiently long, I will look into each lockloss per Sheila's on the phone and alogged advice (and hopefully find with something meaningful to alog).

LOG:

None

DRMI Lockloss unknown reason

ALS and DRMI had already lost lock by the time I was back from making dinner - ALS seems very unstable, which is new issue not seen in the past 5 hrs of locking.

Still Relocking after last NLN lockloss at 23:15 UTC - experiencing a whole host of problems

In the meantime we've had the following issues, with locklosses listed in the lockloss alog 77615:

- BS Glitch and other DRMI locklosses 8 times. The glitch locklosses (4 of these 8 losses) were overcome by waiting at ENGAGE_DRMI_ASC while ASC converges to mitigate the glitch kick. That being said, DRMI is extremely unstable in ENGAGE_DRMI_ASC and lost lock here 2 times right as ASC converged.

- Unknown locklossest at TRANSITION_FROM_ETMX and after initial alignment, LOWNOISE_ESD_ETMX

- IR Not found (also after initial alignment).

I already ran initial alignment and don't think this is the problem. Have troubleshot using Jenne's ppt to no avail. The problem is that I essentially have to wait 1 hr (for the EX Transition issue) before experiencing a lockloss. Having done an initial alignment between EX Transition locklosses, I think it's worth assuming that that didn't fix the issue so I'll resort to the call list when I'm near EX Transition.

Tony, Rahul

ETMX09 was rung up (even though Ibrahim had set the gains off last night) and Tony was trying to fight the Guardian to try and damp this mode early this morning. We were out of Observing (for 15mins while we were figuring it out) and struggling to get in (violins too high message). Hence, at first we set the nominal values of gain in lscparam to zero, then set the violin damping guardian to Damp_Violins_Full_Power and then applied a fraction of the nominal gain. This seems to be slowly damping the mode.

Also then Tony was able to move back to Observing.

Will keep a watch over this mode (and others if needed) until we fully damp it.