Displaying reports 361-380 of 77237.Go to page Start 15 16 17 18 19 20 21 22 23 End
Reports until 12:23, Tuesday 16 July 2024
H1 ISC
sheila.dwyer@LIGO.ORG - posted 12:23, Tuesday 16 July 2024 - last comment - 13:49, Tuesday 16 July 2024(79160)
back to + YAW spot on SR3, back to locking.

Ibrahim, Sheila, TJ, Camilla, Naoki

Since the +/- pitch alignments of SR3 from 79103 both seem like they will not work, this morning I ran intial alignment for the +Y spot again.   After running SR2 align (single bounce, 10W), the power on AS_C NSUM was 0.0219 Watts arriving in HAM6, 5% worse than the 0.023W before Friday (79101). Attached are screenshots of the AS air camera and the sliders after running initial alignment.   After PRMI ASC ran, POP18 NORM was 45, POP90 NORM was 48 counts (sliders and camera in PRMI). 

DRMI locked with POP18 NORM of 50 counts, so still with 15% less power than last Thursday.  We have had a number of locklosses trying to get to DC readout.  We saw that again a DARM Offset of 9e-5 causes the ASC to fail and causes a lockloss, so we changed the DARM offset to 6e-5, and made edits to OMC_LOCK to allow it to lock (lowered prominence in FIND_CARRIER to 5, and height to 6, threshold in OMC_LOCKED to 6 instead of 10).

We have been able to lock, with a range of 100Mpc. 

Editing to add a comparison of this lock to last Thursday's lock. 

The single bounce throughput is (0.219/0.023) = 95% of what it was last Thursday.  The optical gain is then expected to be srt(0.95) = 0.97.  Right now kappa C is reporting an optical gain of 0.907, 1 hour in to the thermalization, so significantly worse than expected.  I think think that this is beyond the range of values where kappa c is a good approximation, so we need to run a calibration measurement soon. 

AS_A, AS_B DC, AS_C, and OMC REFL are 86% of what they were last Thursday.  Power recycling gain and arm powers are still thermalizing, but may be a bit low.  OFI PD A (the one in HAM7, measures rejected beam from the TFP on the side of the OFI closer to HAM6), is lower by 5%, Camilla pointed out to me that the power on this diode changes each time that we move ZM4/5/6, so it's not very reliable for comparisons.  OFI PD B, the one in HAM5 that measures the rejected beam from the first polarizer in the OFI, sees more rejected power by 17% than it did on Thursday. 

We opened the AS beam diverter and captured screenshots of the image with the normal exposure of 800 and with the exposure lowered to 50. 

Images attached to this report
Comments related to this report
keita.kawabe@LIGO.ORG - 13:23, Tuesday 16 July 2024 (79172)

Since we learned about the lower throughput in full lock (didn't know about that before), we're making whatever we can to assess the performance of H1 with this alignment.

However, this means that we will dump more than 10% of power spike into whatever is absorbing power in the next lock loss. I don't think this is a sustainable way to operate H1. I asked Sheila to reduce the power and unlock after we're done with noise characterization. Calibration is NOT going to be done fully as it involves fixing something that doesn't run as of now, we'll have to skip that.

Attached is a comparison of now (Jul 16) VS 8 days ago (Mon Jul 08), both ~2 hours into 60W lock.

channels (all OUT_DQ) Tue Jul 16 Mon Jul 08 Now/Before
ASC-AS_C_NSUM 0.583 0.702 0.830
ASC-OMC_A_NSUM 0.532 0.629 0.846
ASC-OMC_B_NSUM 0.519 0.615 0.844
ASC-AS_A_DC_NSUM 50240 60769 0.827
ASC-AS_B_DC_NSUM 54311 66061 0.822
OMC-REFL_A_LF 571.2 687.4 0.831
Images attached to this comment
thomas.shaffer@LIGO.ORG - 13:49, Tuesday 16 July 2024 (79175)

Same table but with a new column for our SR3 -P alignment from a 17 hour lock on May29. These values were taken the same as above.

channels (all OUT_DQ) May 28 Tue Jul 16 Mon Jul 08 Now/Before
ASC-AS_C_NSUM 0.595 0.583 0.702 0.830
ASC-OMC_A_NSUM 0.571 0.532 0.629 0.846
ASC-OMC_B_NSUM 0.538 0.519 0.615 0.844
ASC-AS_A_DC_NSUM 52000 50240 60769 0.827
ASC-AS_B_DC_NSUM 55383 54311 66061 0.822
OMC-REFL_A_LF 675.6 571.2 687.4 0.831
H1 SEI
ryan.crouch@LIGO.ORG - posted 12:20, Tuesday 16 July 2024 (79166)
H1 ISI CPS Noise Spectra Check - Weekly FAMIS

Closes FAMIS#26000, last checked in alog78993

For the HAMs low frequency noise is reduced

For the BSCs the ETMs look different at low frequency, esp ST2__CPSINF_V{1,2,3}

Non-image files attached to this report
LHO General
ibrahim.abouelfettouh@LIGO.ORG - posted 12:18, Tuesday 16 July 2024 - last comment - 12:41, Tuesday 16 July 2024(79170)
OPS Day Midshift Update

IFO is in NLN and in MAINTENANCE as of 18:26 UTC (52 min lock)

With the +Y position detailed in alog 79164, we were able to achieve NLN.

Taking calibration sweeps right now.

Comments related to this report
ryan.crouch@LIGO.ORG - 12:41, Tuesday 16 July 2024 (79171)
H1 ISC
camilla.compton@LIGO.ORG - posted 12:14, Tuesday 16 July 2024 - last comment - 13:39, Tuesday 16 July 2024(79169)
High DARM coherence with LSC: Feedforward re-tuning needed

We ran the coherence plot (instructions in LowRangeChecks wiki) for 2024/07/16 18:47 UTC with reported 132MPc, plot attached. See lots of coherence with SRCL, PRCL, MICH so we plan to re-tune these feedforwards. The CHARD coherence <15Hz isn't new but the coherence around 30Hz is, it might be coupling through the LSC loops though.

Images attached to this report
Comments related to this report
camilla.compton@LIGO.ORG - 13:39, Tuesday 16 July 2024 (79174)

After turning SRCLFF gain down from 1.14 to 1.0, the coherence looked a lot better and the range increased to 145MPc.

We took FF measurements for SRCL, MICH and PRCL, saved to /opt/rtcds/userapps/release/lsc/h1/scripts/feedforward. Before fitting these we found we didn't like this IFO alignment (79172).

H1 ISC
camilla.compton@LIGO.ORG - posted 11:57, Tuesday 16 July 2024 (79167)
OFI Temperature Control 4x larger than nominal during power up

As looked at in 79117 and 78399. the OFI's temperature control changes as we power up.

In a nominal lock before this alignment change, VOLTAGEBACK changes 0.0091V and the temperature wobbles 0.0031 degC: plot.

In today's power up at the new alignment, VOLTAGEBACK changes 0.0267V and the temperature wobbles 0.0135 degC: plot. This is 3 to 4.5 times larger than usual and larger than the changes we saw May 28th/29th 78442 power-ups: 0.0458V and 0.0097degC plot.

Note that we don't understand if this is damaging at all but it makes us think that we are closer to clipping the OFI or have the beam spot in a different position in relation to the TEC/thermistor: TEC is on bottom of the OFI TGG holder (OMC side): G1902217 slide 22. Unsure location of thermistor.

Images attached to this report
H1 TCS
thomas.shaffer@LIGO.ORG - posted 11:49, Tuesday 16 July 2024 (79168)
TCS Chiller Water Level Top-Off

FAMIS 27793

I added 200mL to the CO2X chiller, but none to CO2Y. The CO2Y filter needs to be replaced at the next opportunity.

All values updated in the T2200289 sheet.

H1 CDS
david.barker@LIGO.ORG - posted 11:06, Tuesday 16 July 2024 (79165)
mains power glitch 10:58:55, lights flickered on site and at my home in Kennewick

CS Mains mon trend attached.

Images attached to this report
H1 CDS (SUS)
ryan.crouch@LIGO.ORG - posted 10:42, Tuesday 16 July 2024 - last comment - 09:11, Wednesday 17 July 2024(78982)
Older medm button generator script revival

At the beheast of Jeff, I have revived this script from 2011 written in python2 to generate medm matrixes. Updating seemed pretty straightfoward, I ran a builtin automated conversion to start, "2to3 file.py" (-w if you want it to actually write the changes).

I tested it by running "python3 generate_KisselButton2024.py 10 4 H1:OMC-ADC_INMTRX" and copy pasting the output into a blank .adl which gives me this 10x4 matrix button with a connected display window based on the input channel arg. It needs a minumum of 3 args: #rows #col channel. You can also specify the size of the button/matrix, and the linked related display .adl file(you can put the whole path).

The new script is found at /opt/rtcds/userapps/release/cds/common/scripts/generate_KisselButton2024.py, and is svned. I left the old script where it is (the same name minus the 2024).

Images attached to this report
Comments related to this report
jeffrey.kissel@LIGO.ORG - 09:11, Wednesday 17 July 2024 (79187)CDS, OpsInfo, SYS
Amazing -- thanks Ryan!
H1 ISC
ibrahim.abouelfettouh@LIGO.ORG - posted 08:42, Tuesday 16 July 2024 (79164)
Fast Shutter Counts and Ratio Before and After Shuttering

AS_A Counts after shuttering yield a ratio of 325 ppm. Find screenshot below. Template for this ndscope is saved under:

/opt/rtcds/userapps/release/sys/h1/templates/ndscope/fastshuttercheck.yaml

This is with the +Y alignment spot in SR3 detailed in alog 79103:

  Start +P move -P move +Y move -Y move
SR3 P slider 438.7 651.1 138.7 438.7 438.7
SR3 Y slider 122.2 122.2 122.2 322.2 -167.8
Images attached to this report
LHO VE
david.barker@LIGO.ORG - posted 08:14, Tuesday 16 July 2024 (79162)
Tue CP1 Fill

Tue Jul 16 08:07:49 2024 INFO: Fill completed in 7min 45secs

Jordan confirmed a good fill curbside.

Images attached to this report
H1 CDS
erik.vonreis@LIGO.ORG - posted 07:51, Tuesday 16 July 2024 (79161)
Workstations updated

Workstations were updated and rebooted.  This was an update to OS packages.  Conda packages were not updated.

LHO General
ibrahim.abouelfettouh@LIGO.ORG - posted 07:33, Tuesday 16 July 2024 (79159)
OPS Day Shift Start

TITLE: 07/16 Day Shift: 1430-2330 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 1mph Gusts, 0mph 5min avg
    Primary useism: 0.01 μm/s
    Secondary useism: 0.05 μm/s
QUICK SUMMARY:

IFO is attempting to lock based on an SR3 +Y position that Sheila believes may have promise. See alog 79103.

Tuesday Maintenance has been deferred for this week as we work on high priority IFO Locking Issues.

Other:

H1:PEM-CS_DUST_PSL102     WARNING: dust counts did not change, please investigate

H1 ISC
sheila.dwyer@LIGO.ORG - posted 07:21, Tuesday 16 July 2024 (79158)
SRY doesn't need LSC triggers

In initial alignment we trigger the length feedback to SRM in SRY.  In the last day I've been setting these thresholds to zero so that we aren't triggering, and this has worked fine many times now.  However, when the alignment is poor and the build ups are too low for the triggers we have been using, the WFS don't work well.  I suspect that we've been using the LSC triggers to make sure the alignment is good enough for the WFS to work. 

We could consider a dither alignment instead of WFS, since the length lock seems very robust.  Or, as a first step let's move the threshold to a check that's run before ASC comes on, rather than preventing length locking of the cavity.

LHO General
tyler.guidry@LIGO.ORG - posted 07:18, Tuesday 16 July 2024 (79157)
End Station Supply Fan Swap
Chris is actively working on refitting our VEA HEPA filters. This required swapping of the fans at the both EX and EY. The swap occurred between 7:10a-7:15a PST.

C. Soike T. Guidry
H1 General (ISC, OpsInfo)
oli.patane@LIGO.ORG - posted 01:28, Tuesday 16 July 2024 (79156)
Ops Eve Shift End

TITLE: 07/16 Eve Shift: 2300-0800 UTC (1600-0100 PST), all times posted in UTC
STATE of H1: Corrective Maintenance
INCOMING OPERATOR: None
SHIFT SUMMARY: I've put the detector in IDLE for the rest of the night. Sheila and I found that the +Y alignment (from 79103) doesn't seem to be the best (see Sheila's thoughts on this alignment: 79155), and so we decided to try out the +P alignment. We were eventually able to get DRMI to lock with this, but the beamspot on AS_AIR didn't look great(attachment1), and once ASC kicked in we started getting mode hopping(attachment2), with DRMI staying locked. We got through to CHECK_AS_SHUTTERS and ran the fast shutter test and got values for ASC_AS_A_DC_NSUM_OUT_DQ of 3.2k before the shutter closed, and ~1 when the shutter closed. giving 250ppm (attachment6). Unfortuately, we lost lock from CHECK_AS_SHUTTERS a while after and I haven't been able to get DRMI locked since then, but PRMI locks quickly.
LOG:

IFO in +Y alignment (79103)
23:00 Troubleshooting and at ENGAGE_ASC_FOR_FULL_IFO
23:07 Lockloss from DARM_OFFSET
23:21 Lockloss FROM CARM_TO_TR
23:49 Got to DARM_OFFSET (offset at 9e-5)
    - 23:50 Dip in power recycling gain started, Ryan S changed darm offset to 6e-5 and was able to save lock, we continued to sit in DARM_OFFSET
23:58 I took ifo to DOWN and into SR2_ALIGN in ALIGN_IFO guardian
    - Comparing AS_{A,B}_NSUM_OUT_DQ and AS_C_NSUM_OUT_DQ values to those channels' values from second to last initial alignment before last NLN on Friday morning and trying to rase those channels' values to match previous alignment (attachment3). horizontal cursors are placed at the bounds where these channels were during the second to last lock's IA.

04:12 Back to DOWN and tried to get DRMI locked
    - locked PRMI, but doesn't look great after ASC (attachment4)
    - DRMI wouldn't lock for more than a second or two, and it looks Bad (attachment5).
04:38 Went to DOWN, decided to try +P location
    - Went into MANUAL_INITIAL_ALIGNMENT, went to SR2_ALIGN, changed power to 10W, and moved SR2 and SR3 to the locations given in 79135. Then Sheila moved SRM to line up the beams on AS_AIR. We went through SRC alignment and offloaded

IFO in +P alignment (79103)
05:24 To DOWN, started relocking
05:39 DRMI locked itself after 10 minutes of trying
    - Looked weird (attachment1)
    - Lots of SRM saturations while in OFFLOAD_DRMI_ASC
    - After ASC, DRMI started splitting while staying locked (mode hopping) (attachment2)
05:42 CHECK_AS_SHUTTERS ran fast shutter test (attachment6)
    - Value in AS_A before shutter close = 3.2k and after = ~ 1.0 counts; 250 ppm overall for ASC_A - not bad, seems like beam spot is fully blocked
        - Compare to Keita's earlier tests 79141, 79147
06:04 Lockloss from CHECK_AS_SHUTTERS, trying to relock
    - Cannot seem to get DRMI to lock again, but PRMI will lock quickly
    - Cycled thorugh PRMI and MICH and still couldn't get DRMI
07:44 Put detector in IDLE for the night

 

Images attached to this report
H1 ISC
sheila.dwyer@LIGO.ORG - posted 22:30, Monday 15 July 2024 (79155)
negative pitch SR3 position seems like it will not work

Oli was searching for a better throughput to AS diodes when I logged in, based on the observation earlier that in the +Y SR3 position from 79103.  As I commented 79153 I think that is explained by poor DRMI build ups.

Because we did have a CARM offset reduction lockloss earlier from the +Y position, I thought we should try a different one, and we opted for -P from 79103. The attached screenshots show the AS camera and slider settings after running SRC alignment, and again after locking PRMI.  The AS camera looks strange in PRMI.  The third screenshot shows the single bounce beam with 10W (after running full initial alignment), it looks clipped on the AS camera. Our conclusion is that there is some clipping or other problem in the -P position, and it isn't a good place to try locking.

Oli has now moved up to the position called positive pitch in 79103, screenshot after running SRC align is attached.

Images attached to this report
H1 ISC
sheila.dwyer@LIGO.ORG - posted 15:43, Monday 15 July 2024 - last comment - 20:58, Monday 15 July 2024(79140)
moved SR3, relocking after fast shutter test

Sheila, Keita, Ryan Short, Camilla and TJ

Part way through initial alignment, we stopped and moved SR3 towards the positive yaw spot from 79103, and used the SR2 osem values from that time.  A small manul adjustment of AS_C was needed, otherwise initial alingment was uneventful.

With DRMI locked, we ran the fast shutter test that Keita and Ryan S have both looked at.

We also looked at the ratios of AS PDs  to compare to   78667, they are most similar to the good times in that alog. After these checks we have decided to try locking.

Comments related to this report
keita.kawabe@LIGO.ORG - 15:55, Monday 15 July 2024 (79141)

Fast shutter behavior is attached. It's working fine, but the throughput to HAM6 is ~14% down compared with before.

Before the shutter was closed, ASC-AS_A_DC_NSUM was ~3.7k counts, and  ~0.75 counts after (fractional number because of decimation to 2k). That's about 200ppm.

However, it used to be ~4.3k and 1cts on one of the "happy" plots alog 79131, 3.7k/4.3k~0.86, so the throughput to HAM6 seems to be ~14% lower than before.

Images attached to this comment
keita.kawabe@LIGO.ORG - 16:33, Monday 15 July 2024 (79147)

IFO lost lock even before going DC and we had another FS test forced by the guardian, and it didn't fail, but the throughput is even worse.

A_DC_SUM=3.4k when the shutter was open, Closed/Open ratio is about 1000ppm, and a tiny part of the beam is being missed by the shutter (attached). Note that I'm NOT eyeballing the "open" Y cursor in log scale, I set it while in linear Y scale but changed it to log to show that the power after the shutter was closed seems to be measurably larger than it should be.

Maybe this is going on because of tiny alignment differences from lock to lock, but anyway this doesn't look to be the place we want.

Images attached to this comment
sheila.dwyer@LIGO.ORG - 20:58, Monday 15 July 2024 (79153)

I think the reason that the power levels were low on AS_A and AS_B in the DRMI lock mentioned above was because DRMI was poorly aligned, not due to excess output arm losses.

Last good lock: DRMI locked, POP18 NORM was 65.3, POP90 NORM was 9.7,  AS_A DC 4173 cnts, AS B DC was 4395 and AS_C was 0.0517 (Watts into HAM6).

In today's earlier DRMI lock, POP18 NORM was 55.35 (15% lower than the good lock) POP90 NORM was 12.27 counts (30% higher).  I think this indicates that DRMI was poorly aligned in this earlier lock, not necessarily that it was a bad spot through the OFI.

H1 ISC
ryan.short@LIGO.ORG - posted 21:16, Saturday 13 July 2024 - last comment - 21:14, Monday 15 July 2024(79103)
Moved SR3 with SR2 following to find spots with good AS port power

To help characterize the recent issues we've seen with IFO alignment that are familiar to the output change in late April, I moved SR3 in 10W single-bounce configuration with the SRC2 ASC loop on so that SR2 would follow (ALIGN_IFO's SR2_ALIGN state), much like in alog77694. I moved SR3 far enough in each direction (+/- pitch and yaw) to bring the ASC-AS_C_NSUM signal back up to our target value of 0.022, which I was successful doing in every direction except -Y, where I could only bring AS_C up to 0.018. In the three directions where I was successful, the beam spot on the AS AIR camera looked much more like the clear circle we're used to seeing and less like the upside-down apostrophe we have now.

It seems that our old output spot (starting place in TJ's alog) is still bad (-Y direction with SR3 from current place) since that was the only direction where I couldn't get the AS_C power back up.

Slider values of SR3 after each move:

  Start +P move -P move +Y move -Y move
SR3 P slider 438.7 651.1 138.7 438.7 438.7
SR3 Y slider 122.2 122.2 122.2 322.2 -167.8

Attachment 1 is the starting AS AIR camera image (and our current spot), attachment 2 is after the +P move, attachment 3 is after the -P move, and attachment 4 is after the +Y move.

Images attached to this report
Comments related to this report
camilla.compton@LIGO.ORG - 15:15, Monday 15 July 2024 (79135)
 
Before
+P move
-P move
+Y move
-Y move
Time
1404952764
2024/07/14 00:39:06 UTC
1404955465
2024/07/14 01:24:07 UTC
1404958133
2024/07/14 02:08:35 UTC
1404959716
2024/07/14 02:34:58 UTC
1404963518
2024/07/14 03:38:20 UTC
H1:SUS-SR3_M1_OPTICALIGN_P_OFFSET
438.7
651.1
138.7
438.7
438.7
H1:SUS-SR3_M1_OPTICALIGN_Y_OFFSET
438.7
122.2
122.2
322.2
-167.8
H1:SUS-SRM_M1_DAMP_P_INMON
-1033.7
-1035.2
-1036.3
-1036.1
-1037.1
H1:SUS-SRM_M1_DAMP_Y_INMON 
913.7
914.0
914.1
914.1
914.2
H1:SUS-SR2_M1_DAMP_P_INMON
597.7
-871.6
2660.2
614.7
566.2
H1:SUS-SR2_M1_DAMP_Y_INMON 
1125.3
1179.9
1069.3
1878.8
-72.4
H1:SUS-SR3_M1_DAMP_P_INMON 
-290.2
-57.7
-619.4
-297.9
-279.1
H1:SUS-SR3_M1_DAMP_Y_INMON
-411.0
-425.2
-390.4
-256.9
-633.7

Here are the OSEM values, so that Alena can cotinine her 78268 analysis of the OFI beam spot position.

sheila.dwyer@LIGO.ORG - 21:14, Monday 15 July 2024 (79154)

Adding a screenshot of sliders for the -P alignment above, after initial alignment.

Images attached to this comment
H1 SYS
daniel.sigg@LIGO.ORG - posted 20:27, Saturday 13 July 2024 - last comment - 12:00, Tuesday 16 July 2024(79102)
Fast shutter is OK

Yesterday, the fast shutter test failed due to dark offsets in the AS WFS DC NSUM channels.

The guardian started running the TwinCAT testing code, which works as intended: it sends a close command to the trigger logic, which in turn fires the fast shutter. The fast shutter works fine as can be seen on the HAM6 geophones. The slow controls readbacks also indicate that both fast shutter and PZT shutter are closed no later than 200ms after the trigger. However 0.5sec after the guardians started the test, it checks the AS WFS DC NSUM outputs and compares them against dark offset limits of ±15. Since the dark offset on WFS B was over 30, the guardian then sent an abort command to the TwinCAT code and reported a failure.

Comments related to this report
keita.kawabe@LIGO.ORG - 23:35, Saturday 13 July 2024 (79104)

That story agrees with my observation on Friday night when I started looking at the FS after 11:15 PM.

Sheila reported (79092) that the Fast Shutter reopened only after ~50ms or so. It seems that the low voltage drive to keep the shutter closed was not working. 1st attachment shows the last time that happened at around 23:10 local time. Daniel points out that the shutter was in an error state at that time but that was after Ryan power cycled the FS driver. We don't know exactly what kind of state the fast shutter was in here.

The next time the HV firing was tested was at 23:23 local time (2nd attachment), the shutter was kept shut (i.e. low voltage thing was somehow working) but there are two things to note.

  1. H1:ASC-AS_B_NSUM_OUT_DQ was about 25 or so, i.e. larger than the dark offset threshold of +-15. The FS guardian go to failed state.
  2. Bouncing was worse than I've ever seen. It bounced multiple times with 40 to 50ms period and eventually settles to the "closed" posotion. (Before Friday I only saw single bounce and that was it.)

The last FS test I've done was 00:12 local time on Jul/13 when the error was cleared, with the smaller power than nominal (3rd attachment). Bouncing was as bad but the power coming to HAM6 was smaller (see the trigger power at the top left). AS_B_NSUM was somewhat smaller (more like 10).

The reason why AS_B_NSUM is worse is because I reduced the analog DC gain by a factor of 10 and compensated for that by digital gain. The effect of analog offset as well as ADC/electronics noise are 10x worse than AS_A. I adjusted the dark offset while IMC was unlocked but we can probably increase the threshold to 30 or so if it continues to bother us.

Bouncing behavior might be more serious as it could mean that the beam was close to the end of the travel of the FS mirror (and it was bad on Friday because of funny alignment), or low voltage drive was somehow still funny. I suspect the former.

Images attached to this comment
ryan.short@LIGO.ORG - 17:25, Monday 15 July 2024 (79126)

Here is my attempt at recreating the (rough) timeline of events related to the fast shutter Friday night (all times in UTC):

  • 04:55:41 - While locking, ISC_LOCK gets to CHECK_AS_SHUTTERS and has the FAST_SHUTTER Guardian go to its TEST_SHUTTER state
    • The fast shutter test runs. Shutter closes, but due to improper dark offsets in the AS WFS DC NSUM channels (noted by Daniel above), Guardian thinks the test failed
      • Guardian sets AS_PROTECTION error code
      • Guardian reports "Fast shutter failed tests!! Downrange light does not disapear properly!" and jumps to SHUTTER_FAILURE
  • 04:55:43 - The fast shutter shows its error code
  • 04:55:46 - Fast shutter error code clears
  • ~05:20 - I power cycle the fast shutter driver chassis
  • 05:21:46 - Shutter opens
  • 05:28:00 - Next shutter close test; only closes for 65ms
  • ~05:45 - I power cycle the shutter logic chassis
    • No change in behavior is seen
  • 06:10 - There have been several attempts at running the shutter test at this point, all of them the shutter only was closed for 60-70ms at a time
  • 06:20 - The shutter finally closes and stays closed
  • 06:53 - Keita fixes the dark offset on AS WFS
  • 07:10 - Keita uses the Guardian to test the shutter; it works and stays open
  • 07:12 - Beckhoff errors clear after another full cycle using the Guardian
Images attached to this comment
keita.kawabe@LIGO.ORG - 14:53, Monday 15 July 2024 (79131)

Seems like ASC-AS_B_DC gain was probably a red herring, important thing is that the beam got uglier/bigger at some point, therefore a part of the beam was not blocked by the fast shutter.

First attachment shows when ASC-AS_B_DC gain switch was flipped from x10 to x1 on Tuesday. You can see that the Fast Shutter has been firing OK until Friday evening.

The rest of the plots show the FAST Shutter test done by the Beckhoff at three different points in time, i.e. the last test before my AS_B_DC change ("happy" Monday July/08 ~16:37 UTC or 9:37 local), the first one after my AS_B_DC change ("happy" July/11 4:43 UTC or Wed July/10 21:43 local time), and the first time the FAST shutter went into the error mode ("sad" Jul/13 4:56 UTC or Fri Jul/12 21:56 local time). The last one is when Sheila and Ryan started having problem.

Important points are:

  1. The two "happy" plots are quite similar. Especially, even though I see bouncing action, the power measured by both AS_A and AS_B more or less settle to the minimum value. Ratio of closed/open is about 230ppm in both cases. This is compatible with the specification of Fast Shutter mirror (which is <1000ppm).
  2. The "sad" plot looks nothing like the happy ones. Right after the shutter was closed the light level went down to less than 1000ppm, but bounced back and eventually settled down to ~6500ppm. You can also see that the light level when FS was open (~2200 cts) is about 50% of what it used to be in happy plots.
  3. ASC-AS_C was pretty close to center in all of the three plots.

From these, my conclusion is that the beam position on the fast shutter mirror was pretty much the same in all three tests, but the beam was pretty ugly for the "sad" plot as was witnessed by many of us on AS port camera. Because of this a part of the lobe was missed by the Fast Shutter. Centering an ugly beam on AS_C might have complicated the matter.

Later, when I forced the test with much lower power, the error was cleared because even though the ugly beam was still there the power went lower than the "shutter closed" threshold of the guardian.

I don't fully understand who did what when during the time the shutter was in the error state (it includes people pressing buttons and power cycling the driver and then pressing buttons again, and I certainly pressed buttons too).

Looking at this, and since Daniel agrees that the Fast Shutter has been working fine, my only concerns about locking the IFO are:

  1. The beam should be properly attenuated by the fast shutter. Not thousands of ppm, but more like ~230ppm.
  2. The beam should not clip on ASC-AS_C. If it's not clipped there, the chances of thing getting clipped downstream is very small.
Images attached to this comment
keita.kawabe@LIGO.ORG - 12:00, Tuesday 16 July 2024 (79148)

FYI, fast shutter has never failed to fire up and never failed to keep shutter closed when the trigger voltage crossed the threshold of 2V since I connected the ASC-AS_C analog sum  to the PEM ADC to monitor the trigger voltage at 2kHz at around Jul/11 2024 18:30 UTC, i.e. in the past 5 days.

In other words, FS has never failed when we actually needed it to protect AS port.

In the 1st attachment, top is the trigger voltage monitored by the ADC. Red circles indicate the time when the trigger crossed the threshold. We've had 12 such events, including 4 that happened after Friday midnight local time.

Middle plot shows when the fast shutter was driven with high voltage (sensed by GS13). Bottom shows the AS_A_DC_NSUM and B_DC_NSUM.

The reason why it looks as if the trigger level changed before and after t=0 on the plot is because a 1:11 resistive divider with a total resistance of 10kOhm was installed at t=0. Before that, the 2V threshold was 32k counts, 2980 counts after.

The rest of plots show that each and every time the shutter was triggered by the trigger voltage rather than a test, it always fired up and kept the shutter closed.

Images attached to this comment
Displaying reports 361-380 of 77237.Go to page Start 15 16 17 18 19 20 21 22 23 End