WP11753
Jonathan, Dave:
As part of the investigation into recent camera image instabilities we upgraded h1digivideo3's 106 VLAN network port from 1GE copper to 10GE fiber. A solar flare PCIe card was installed in h1digivideo3 at 14:05 this afternoon.
The new fiber is connected to a spare 10GE port on sw-msr-h1aux (ethernet 3/1/2).
On h1digivideo3 we have left the original copper connection to eno2, the new fiber port is enp1s0f0np0.
Currently the EPICS_LOAD_MON channel being trended by the DAQ is still H1:CDS-MONITOR_H1DIGIVIDEO3_NET_RX_ENO2_MBIT, the new channel is H1:CDS-MONITOR_H1DIGIVIDEO3_NET_RX_ENP1S0F0NP0_MBIT, which Jonathan is working on remapping to ENO2 so we don't need a DAQ restart.
As we went through SDF revert yesterday when locking all the OMC SDFs were reset. I turned off the QPD A offsets and reset the input and output ASC matrices to those we had found allowed us to lock the OMC with ASC and no saturation of the OMC suspension.
First picture is the reference picture from yesterday before the revert. I have not set back the changes to DCPD offsets as I think these were to make sure DCPD SUM output did not dip to a negative number due to dark noise.
Second picture is the OMC model SDFs now.
The guardian has been set to not go through SDF revert when we lose lock however the guardian has the ASC gains hard-coded in so we may want to replace these with the new values in the OMC guardian once we verufy these are correct by manually locking the OMC once the full IFO is locked.
I accepted these OMC model values above in SDF. Picture attached.
The new POS_X and ANG_Y gain values (accepted in previous comment's screenshot) have been updated in the OMC_LOCK Guardian's ASC_QPD_ON state (where they are hard-coded in). Changes loaded and updated in svn.
As we aligned into the OMC before locking efforts started and we think this was a good OMC alignment I accepted these alignment slider values for OM1,2,3 and OMC in SDF, picture attached.
Locking woes continue. We reran the baffle align scripts for all 4 quads this morning and after a few touch ups, ALS now seems to be in a good spot. Commissioners are now trying to get through DRMI locking as flashes on our pop signals are not great. A more detailed alog to follow.
Per commissioner request, I've made two changes to the early main locking steps as set in ISC_LOCK:
Changes have been loaded and committed to svn.
I've also commented out SQZ_MANAGER from the list of managed nodes in ISC_LOCK. This allows SQZ to work independently without main IFO locking telling SQZ_MANAGER what to do for now.
EDIT: We later learned that lines 214-215 of ISC_LOCK needed to be commented out as well since this is a request of SQZ_MANAGER in the DOWN state.
Furthering this effort as main IFO locking is progressing, I've commented out the first couple lines in the LOWNOISE_LENGTH_CONTROL state which interacts with SQZ_MANAGER, which at this point is not managed.
Naoki, Vicky - We have undone these changes guardian changes for the break (brought back SQZ_MANAGER in list of managed nodes, in first few lines of LOWNOISE_LENGTH_CONTROL, and lines 214-215 requesting sqz to down).
SQZ_MANAGER is back to being managed in ISC_LOCK as usual. We will see the lock sequence through a few times and get it running smoothly, and update on that after relocking.
Wed Mar 06 10:14:59 2024 INFO: Fill completed in 14min 55secs
Previously done 75898 - closes FAMIS#27784
Both TCSX and TCSY were at their max fill level, so I did not add any water. There was no leaking.
Patrick, Jonathan, Erik, Dave:
We are working on an issue with some cameras on the new video server (h1digivideo3) becoming unstable over the past few weeks, resulting in their images going blue-screen until their server process is restarted.
I've opened FRS30615 for this issue.
As part of this investigation, yesterday we removed the first camera service from h1digivideo3 (FC TRANS-A IR) because its camera is not working. We were hoping reducing the cpu loading would make the other camera services more stable.
This resulted in the EDC disconnecting from the 20 channels associated with this camera. To "green up" the EDC, I am running a dummy IOC on cdsioc0 which serves these chans (tmux session called cam_fc_trans_a_dummy_ioc)
We also rebooted h1digivideo3, which had been up 190days.
Neither of these fixed the problem.
TITLE: 03/06 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Preventive Maintenance
OUTGOING OPERATOR: None
CURRENT ENVIRONMENT:
SEI_ENV state: CALM
Wind: 1mph Gusts, 0mph 5min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.16 μm/s
QUICK SUMMARY:
H1 still struggling with locking post vent. Troubleshooting continues, starting off the morning with an initial alignment.
Naoki, Daniel, Nutsinee
The newly installed squeezer (iLIGO) PMC took a while to lock so we investigated. Turned out the PMC had difficulty locking when the PZT was scanning from 0-100V but was able to lock reliably when the PZT turned around from 100-0V. We maybe seeing something we've seen before in the OPO where some element inside of the PMC heated up, expanded the cavity length against the PZT movement from 0-100V making it difficult to grab a lock. We triggered the PMC lock on the Refl diode and the channel we used has 2kHz sampling rate. Beckhoff trigger is running at 1kHz. The resonance passed by so quickly that sometimes if we're lucky the trigger would work but most of the time it didn't.
However, when scanning from 100-0V the cavity expansion help extended the period that light stored inside the cavity, making it easier to grab lock. Now that the PMC is better aligned, the effect became more severe.
The scan shows PMC refl and trans having a much narrower dip/peak when scaning from 0-100V compared to 100-0V.
We revert the scan from 0-100V to 100-0V so it starts from the top. Changes related to PMC scan has been accepted in the SDF.
In case of the OPO the cavity self-lock due to heating within the crystal. In case of PMC we have about ~100mW of light missing that can't be accounted for. We believe this amount of light maybe absorbed somewhere inside of the PMC.
Elenna, Jenne, Gabriele, Matt, Trent, Louis, Georgia
- Jenne and I checked the fast shutter and ran the fast_shutter test (SYS -> AS port protection -> run)
- After the initial alignment (alog here) we got through green arms locking at up to CHECK_MICH_FRINGES. When we tried to do PRMI there was a timer that kept taking it back to MICH fringes since the POP90 buildups were so poor. On the AS_AIR camera the alignment looks bad in yaw.
- We edited ISC_LOCK line 1353 to lower the threshold for PRMI flashes on POPAIR_B_RF90 from 20 to 4.
- We ran the dark offsets script with the IMC offline and the ALS shuttered (sdf screenshots here), and started locking again.
- The Y arm was finicky, we struggled to get the flashes above 0.6... We lowered the PD threshold in the ALS Y PDH (H1:ALS-Y_REFL_LOCK_TRANSPDNORMTHRESH) from 0.7 to 0.6.
- We also lowered the threshold in ALS_ARM gen_INCREASE_FLASHES from 0.9 to 0.6.... feels like cheating, maybe the alignment onto ISCT1 is off? The camera alignment for ALS_Y also looks bad (see screenshot)
- Even though we did an initial alignment earlier today, the PRMI flashes are bad and the MICH fringes don't look great. PRMI_ASC caused a couple of locklosses, so we took the guardian to PRMI_LOCKED and tried aligning it by hand. Elenna aligned PRM and BS, but the buildups and camera indicated bad pitch alignment.
- We re-ran initial alignment, and then the guardian took us straight up to PRMI_ASC, DRMI also locked easily! POPAIR_RF18 was a little noisy.
- We started the CARM offset reduction sequence but lost lock at DHARD_WFS, the two arms were very different (higher build up for Y arm), when DHARD came on the arms were pulled closer together but we lost RF9, and we lost lock.
- We decided to stop at DARM_TO_RF, step the X arm (X hard) closer using the move_arm_dev script and fix PRM and SRM alignment to increase the buildups.
- While we stepped the arm and adjusted PRM, the green arms lost lock. We think we might be aligning the whole IFO to some bad ITM alignments (how accurately aligned are they after the baffle PD alignment script?)
We decided to call it a night at this point.
A other things that we found during the inital alignment process:
We changed the threshold in ALS_ARM gen_INCREASE_FLASHES back to 0.9
We are thinking of increasing the guardian timer for INCREASE_FLASHES but have not done so yet.
We also reverted this change, since DRMI flashes are back to their usual level:
We edited ISC_LOCK line 1353 to lower the threshold for PRMI flashes on POPAIR_B_RF90 from 20 to 4.
Sheila, Gabriele, Corey
Summary of guardian changes that are useful to make when we are recovering from a vent, for future reference:
Some guardian changes made yesterday to think about maybe reverting:
We lost dmt displays in the control room around 12:16 today. John Zweizig and I have been working on this for the past few hours. What seems to have happened is after a reboot h1dmt1 is having problems with redundant data streams (for reference the identically configured h1dmt2 has no issue). When we disable one of the input streams it is able to receive data. At this point we have disabled one of the streams and will look at it more in the morning.
Matthew, Jennie W, Gabriele
In the initialization of the OMCscan code (which gets OMC scan data, analyzes it and then plots it), I updated several values to reflect the transition from OMC003 to OMC001, so that omc analyses are accurately done. For example, several small changes include:
The values were obtained from T1500060 Table 19, which report the OMC optical test results for OMC001; note: the conversion from nm/V to MHz/V is found by the relation delta(f)/f = delta(L)/L, where delta(L) is 2*PZTresponse in nm/V, L is the round-trip cavity length, and f is 1064nm converted to MHz
Are we sure that the previous OMC used OMC's "PZT2" (12.7nm/V) for the scan, not OMC's "PZT1" (11.3nm/V)?
I mean: there is a possibility that the indication of PZT2 on the screen may not mean PZT2 on the OMC.
Also the response of the PZT is nonlinear and hysteretic.
I'd rather believe the frequency calibration using the cavity peaks (e.g. FSR/Modulation freqs) than the table top calibration of the PZTs.
Good suggestion!
Computing the PZT response from the FSRs we get around 6.3 MHz/V.
And on your note about certainty of using PZT2 response, I am not sure.
I think we usually used the channel PZT2 to perform scans with OMC 003. But yeah, I am not sure if this corresponds to PZT2 on the real OMC. The PZT calibration we just use in the scan analysis to get an initial guess for the calibration but the final calibrated scan does indeed find the carrier 00 and 45 MHz 00 peaks to fit the non-linearity of the PZT.
I made a LOCK_PMC state in SQZ_MANAGER guardian. The LOCK_PMC state is between LOCK_TTFSS and LOCK_SHG states. The LOCK_PMC state is copied from LOCK_SHG state, but I commented out the PZT checker because the PMC_PZT_OK function is not defined now.
Camilla, Nutsinee
SQZ_MANAGER now takes charge of PMC Guardian. For now we commented out FC related activities in LOCK_OPO_AND_FC state of the SQZ_MANAGER and LOCKING_HD state in SQZ_LO_LR. Look for "NK March 6" for any changes made today in SQZ_MANAGER and SQZ_LO_LR Guardian. These should be reverted when we have the filter cavity back. For now we can lock PMC all the way to LO with SQZ_MANAGER and will automatically relock themselves when IFO kills the PSL.
We have filter cavity. Changes have been reverted.