daniel.sigg@LIGO.ORG - posted 20:27, Saturday 13 July 2024 - last comment - 12:00, Tuesday 16 July 2024(79102)
Fast shutter is OK
Yesterday, the fast shutter test failed due to dark offsets in the AS WFS DC NSUM channels.
The guardian started running the TwinCAT testing code, which works as intended: it sends a close command to the trigger logic, which in turn fires the fast shutter. The fast shutter works fine as can be seen on the HAM6 geophones. The slow controls readbacks also indicate that both fast shutter and PZT shutter are closed no later than 200ms after the trigger. However 0.5sec after the guardians started the test, it checks the AS WFS DC NSUM outputs and compares them against dark offset limits of ±15. Since the dark offset on WFS B was over 30, the guardian then sent an abort command to the TwinCAT code and reported a failure.
That story agrees with my observation on Friday night when I started looking at the FS after 11:15 PM.
Sheila reported (79092) that the Fast Shutter reopened only after ~50ms or so. It seems that the low voltage drive to keep the shutter closed was not working. 1st attachment shows the last time that happened at around 23:10 local time. Daniel points out that the shutter was in an error state at that time but that was after Ryan power cycled the FS driver. We don't know exactly what kind of state the fast shutter was in here.
The next time the HV firing was tested was at 23:23 local time (2nd attachment), the shutter was kept shut (i.e. low voltage thing was somehow working) but there are two things to note.
The last FS test I've done was 00:12 local time on Jul/13 when the error was cleared, with the smaller power than nominal (3rd attachment). Bouncing was as bad but the power coming to HAM6 was smaller (see the trigger power at the top left). AS_B_NSUM was somewhat smaller (more like 10).
The reason why AS_B_NSUM is worse is because I reduced the analog DC gain by a factor of 10 and compensated for that by digital gain. The effect of analog offset as well as ADC/electronics noise are 10x worse than AS_A. I adjusted the dark offset while IMC was unlocked but we can probably increase the threshold to 30 or so if it continues to bother us.
Bouncing behavior might be more serious as it could mean that the beam was close to the end of the travel of the FS mirror (and it was bad on Friday because of funny alignment), or low voltage drive was somehow still funny. I suspect the former.
Here is my attempt at recreating the (rough) timeline of events related to the fast shutter Friday night (all times in UTC):
Seems like ASC-AS_B_DC gain was probably a red herring, important thing is that the beam got uglier/bigger at some point, therefore a part of the beam was not blocked by the fast shutter.
First attachment shows when ASC-AS_B_DC gain switch was flipped from x10 to x1 on Tuesday. You can see that the Fast Shutter has been firing OK until Friday evening.
The rest of the plots show the FAST Shutter test done by the Beckhoff at three different points in time, i.e. the last test before my AS_B_DC change ("happy" Monday July/08 ~16:37 UTC or 9:37 local), the first one after my AS_B_DC change ("happy" July/11 4:43 UTC or Wed July/10 21:43 local time), and the first time the FAST shutter went into the error mode ("sad" Jul/13 4:56 UTC or Fri Jul/12 21:56 local time). The last one is when Sheila and Ryan started having problem.
Important points are:
From these, my conclusion is that the beam position on the fast shutter mirror was pretty much the same in all three tests, but the beam was pretty ugly for the "sad" plot as was witnessed by many of us on AS port camera. Because of this a part of the lobe was missed by the Fast Shutter. Centering an ugly beam on AS_C might have complicated the matter.
Later, when I forced the test with much lower power, the error was cleared because even though the ugly beam was still there the power went lower than the "shutter closed" threshold of the guardian.
I don't fully understand who did what when during the time the shutter was in the error state (it includes people pressing buttons and power cycling the driver and then pressing buttons again, and I certainly pressed buttons too).
Looking at this, and since Daniel agrees that the Fast Shutter has been working fine, my only concerns about locking the IFO are:
FYI, fast shutter has never failed to fire up and never failed to keep shutter closed when the trigger voltage crossed the threshold of 2V since I connected the ASC-AS_C analog sum to the PEM ADC to monitor the trigger voltage at 2kHz at around Jul/11 2024 18:30 UTC, i.e. in the past 5 days.
In other words, FS has never failed when we actually needed it to protect AS port.
In the 1st attachment, top is the trigger voltage monitored by the ADC. Red circles indicate the time when the trigger crossed the threshold. We've had 12 such events, including 4 that happened after Friday midnight local time.
Middle plot shows when the fast shutter was driven with high voltage (sensed by GS13). Bottom shows the AS_A_DC_NSUM and B_DC_NSUM.
The reason why it looks as if the trigger level changed before and after t=0 on the plot is because a 1:11 resistive divider with a total resistance of 10kOhm was installed at t=0. Before that, the 2V threshold was 32k counts, 2980 counts after.
The rest of plots show that each and every time the shutter was triggered by the trigger voltage rather than a test, it always fired up and kept the shutter closed.