Mon Nov 06 10:08:39 2023 INFO: Fill completed in 8min 35secs

Travis confirmed a good fill curbside.

Summary of the report:

• H1 has been in observing mode for an average of 55.47% of the time this week.
• Maintanence was performed on Tuesday which caused some of the downtime for that day.
• Some glitches seen this week have correspondence to maintanence performed the week before mentioned in alog_73852.
• Horizon lines can be seen on Monday, Tuesday, Thursday, Saturday, and Sunday. 
• Toward the end of the week it should be noted that there is a high frequency of microseism, and it seems to be steadily increasing.
• Saturday currently has unexplained lockloss at 4:12 UTC ( I will check back on this later for an explanation).
• Fscan plots appeared everyday except for Wednesday, which only had an observing percentage of 4.43%. On the other days the counts were all within normal ranges.
• There were some winning channels in Hveto that had not appeared in some time (recent weeks) such as : H1:ASC-Y_TR_A_NSUM_OUT_DQ, H1:HPI-HAM4_BLND_L4C_Z_IN1_DQ, and H1:HPI-ETMX_BLND_L4C_RX_IN1_DQ
• High SNR glitches seen on Monday, Tuesday, Friday and Saturday.

This is a link to the full report.

TITLE: 11/06 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 153Mpc
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 3mph Gusts, 2mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.18 μm/s
QUICK SUMMARY: Locked for 9.5 hours. No current alarms, temps are very stable. Our range looks to be slowly decreasing, I'll look into it.

TITLE: 11/06 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 162Mpc
INCOMING OPERATOR: Ryan C
SHIFT SUMMARY:

• 00:12 Started observing
• 05:33 Lockloss - alog
• 06:21 Reached NLN, fully automated relock
  • CO2 powers still not nominal, same issue as mentioned in 74012
• 06:32 Started observing

LOG:

No log for this shift.

Lockloss @ 05:33 UTC - no obvious cause.

State of H1: Observing at 157Mpc

H1 has been locked and observing for just over 4 hours. Quiet evening so far; microseism is slowly falling.

Genevieve, Lance, Tyler, Robert

In August, we determined that the 52 Hz peak in DARM could be dramatically reduced or eliminated by turning off the chilled water pump CHWP-2 at EX (72331), which was running at 52 Hz. On Friday I decided to check if the water pump was lining up in frequency with a mechanical resonance that made the coupling worse. So I used the variable frequency drive to move the pump frequency around. The figure shows that I found three frequencies where the water pump did not produce an obvious peak in DARM and one where it produced a bigger peak in DARM.

After consulting with Tyler, I have left the frequency at 58 Hz (98% on the FMCS screen). Of course whatever resonance was associated with the peak in DARM at 52 Hz will still be excided by non-pump seismic motion, so the 52 Hz peak may not go away completely. I may move the pump frequency around some more this week to better understand the coupling.

TITLE: 11/05 Eve Shift: 00:00-08:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
OUTGOING OPERATOR: Austin
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 11mph Gusts, 9mph 5min avg
    Primary useism: 0.03 μm/s
    Secondary useism: 0.29 μm/s
QUICK SUMMARY: H1 just reached NLN and waiting for ADS to converge.

• All systems look good; microseism is still elevated

TITLE: 11/05 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Lock Aquisition
INCOMING OPERATOR: Ryan S
SHIFT SUMMARY:

- Started off with running an IA, which ran through with no issue

• Lockloss at FIND_IR (COMM beanote was at -12, DIFF at -6), IMC unlocked, then again at LOCKING_ALS, same reason...
• I noticed that ALS DIFF was VERY noisy when trying to acquire IR, it was at its resonant peak but would not actually catch and just oscillate around it, despite me trying to manually walk it
  • I would assume this oscillation of DIFF and it not REALLY being found caused issues that Ryan saw last night once they got further into the locking sequence - scope attached
• Tried running through CHECK SDF to see if maybe a bad offset was implemented, but ran into the same issue
• The IMC also is continuously losing lock during reaquisition, unsure why
  • Since the IMC is having issues as well, I have attached witness sensors for PR2/3/M as well as MC1/2/3 just to be thorough, seems like there is some shift in MC2/3 and PR2, but I'm not entirely convinced it is a drift/alignment issue (especially since we can lock COMM and ALS with relative ease)

- Went through the rotating call list: contacted Daniel, left a message, then called Jim who noticed that all 4 quad ISI SC guardians had not been reset since the master timing was swapped

• This fixed the issue, these SC offsets being off for all quads coupled with high microseism (which can cause issues during CARM offset stages according to Jim), made it near impossible to lock H1
• To prevent this from happening in the future, Jim pointed out that if we ever do a reboot of any of the front ends containing any BSC ISI to take SEI ENV to MAINT then back to CALM once the front ends come back up - his alog here

- After the issue was solved, relocking was automated, back to NLN @ 19:47/OBSERVE @ 20:10

• SQZ MANAGER message: "SQZ ASC AS42 not on?? Please RESET SQZ ASC"
  • Cycled the guardian which fixed the issue
• TCS POWER message on DIAG MAIN: "CO2 powers are not in nominal state"
  • However when I checked the TCS guardians, all the guardians were reading back green and happy - not sure why this message is showing but was still able to go into observing nonetheless - Tagging TCS
  • Attached ss of ethercat TCS channels - not sure if these are nominal so leaving for team TCS to look at on Monday
  • Also attaching CO2 overview for both arms

- H1 went into COMMISSIONING @ 21:50 to adjust SQZ temperature/check SQZ angle - attached SDF

• Back to OBSERVE @ 22:04

- Lockloss @ 22:58, cause unknown

- Relocking:

• ALS DIFF still looks wobbly but was high enough to catch and move forward without intervention
• Went through PRMI locking
• Rest of locking was automated, back to NLN @ 23:57

LOG:

No log for this shift.

Lockloss @ 22:58, looks like ASC-AS_A_DC saw motion first. Also noticed that the IMC unlocked right before the lockloss (does this usually happen?)

H1 has just back it back to NLN (@ 19:47 UTC), after 18 hours of being down due to issues relocking. Problem was diagnosed to be a culmination of high microseism and the ISI SC guardians not being in the right state for all 4 quads after the master timing was swapped on Friday. The ISI SC issue has since been fixed and microseism is finally starting to trend down.

Sun Nov 05 10:10:16 2023 INFO: Fill completed in 10min 13secs

Last night, while LLO was down and before we lost lock, I used 1/2 hour to inject 13.1 Hz ground currents onto the chasis of AC1 in the CER, and also imitated the 13.1 Hz sound AC1 made using the large speaker. Neither the acoustic nor the electronic ground injections produced noise in DARM at the same level as AC1 had.

TITLE: 11/05 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Aligning
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
    SEI_ENV state: SEISMON_ALERT
    Wind: 5mph Gusts, 4mph 5min avg
    Primary useism: 0.05 μm/s
    Secondary useism: 0.43 μm/s 
QUICK SUMMARY:

- Issues still ongoing from last night with a bad COMM/locklosses at TR_CARM, will start trying to troubelshoot

- Microseism is on the rise, making ALS hard to lock

- CDS/DMs ok

Lots of issues keeping GREEN_ARMS locked (maybe from microseism) and finding IR (keeps missing DIFF), we're unable to survive past CARM_TO_TR. The common mode servo board output signals don't look abnormal, there was some glitching but trending back it usually does this. I tried holding at START_TR_CARM and PREP_TR_CARM and we can survive, I looked at a bunch of the signals and they all seemed normal, MICH, ADS, IMC/LCS REFL_SERVO. When I tried to move on I got the NO IR in arms and we lost lock. Other times in the same state we lost lock at various stages of the CARM gain reduction.

Both the arms have been pretty unstable the whole shift (Y more so), but the wfs has managed to keep it locked for a decent amount of the time, it got worse towards the end. While the microseism is high we've been able to lock with it at the same levels before so it's probably not the sole issue but it certainly isn't helping.

I've done some injections today but I'll alog about that later.

After I was done with the injections, the breakout board was removed from the back of the OM2 heater driver and the Beckhoff cable was fully connected again. This happened at around 21:40 UTC today.

Yesterday during the maintenance, Fernando and Daniel changed the Beckhoff configuration such that the Beckhoff terminal won't switch between thermistor 1 and thermistor 2 (alog 73886). Now it's only reading thermistor 1 (cold one) w/o switching. Thermistor 2 (hot one) is not read out.  Detchar, please see if you can see the comb.

Took MICH FF measurements following instructions in lsc/h1/scripts/feedforward/README.md but running MICHFF_excitation_ETMYpum.xml. Data saved in lsc/h1/scripts/feedforward/ as .txt files. Last tuned MICH in 73420. 

I saved a new filter in FM6 as 27-10-23-b (red trace) but it made the MICH coupling worse between 20 and 80Hz so we left the original (pink trace). We could try to re-fit the data to load in Wednesday's commissioning period.