Oli, Ibrahim

IFO has been swept.

Of note:

• Crane had to be re-parked to right spot.
• Scissor lift on pallet jack (pictured below) was found plugged in and unplugged. Tagging FMCS.
• Unplugged extension cord (was plugged in with nothing plugged into it).

Per the WP12246:

Visual inspections were performed in the LVEA to track the wires and verify the picomotor controllers existence, connections and spares (physically only). The information gattered was updated in the document E1200072. Important findings were found and the document is pretty much near to the real installation. Electrical part investigations will follow to determine the spares. The names of the rack as well as the physical location for the controllers were verified using the O5 ISC wiring diagram D1900511.

Marc, Fernando

WP12239 New VM server

Erik, Jonathan:

The old proxmox machine was taken out of service and repaired. In production it was replaced with a W2275 (aka oaf FE). Please see Jonathan's alog for details. After h0epics was moved, its iocs were started by hand as per wiki instructions:

dust monitors (lvea, lab, ey, ex, dr), weather stations (ex, ey, mx, my) and 3ifo dewpoint sensor.

WP12236 New RGC and Timing Card h1omc0

Jonathan, Erik, EJ, Marc, Daniel, Dave:

h1omc0 was upgraded to a custom RCG5.30 and a new LIGO Timing Card (ver1589). This will permit changing the Duotone frequency from 960,961Hz to 1920,1921Hz in conjection with LLO next week. Erik verified the duotone is currently unchanged at 960,961Hz.

Please see Erik's alog for details.

The RCG upgrade of h1iopomc0 added one slow channel to the DAQ INI (H1:FEC-179_TIMING_CARD_TEMP_DEG_C). A DAQ restart was required, which permitted an EDC change for several pending WPs (marked as *).

WP12243 Slow Controls Timing Card Version Check

Daniel

Daniel changed the slow-controls Beckhoff code to permit the following versions of the LIGO timing card in the FE IO-Chassis

The code was restarted at 10:19. This fixed the persistant EX timing error due to h1susex TCver=1000 and preempted a corner station h1omc0 error with TCver=1589.

WP12164 Add new Guardian PCALX_STAT channels to DAQ*

Tony, TJ, Dave:

The new PCALX_STAT guardian node's DAQ channels were added to the EDC as part of the DAQ restart.

WP12195 Add missing CDSRFM slow channel to DAQ*

Dave:

A new H1EPICS_CDSRFM.ini was loaded, adding the missing channel H1:CDS-RFM_LRS_EX2CS_CHCNT

WP12185 add VACSTAT slow channels to DAQ*

Dave:

A new H1EPICS_VACSTAT.ini added the new state channels to the DAQ.

CDS Hardware Status IOC

Erik, Dave:

Following the upgrade of h1omc0's timing card the CDS HW stat reported problems with this card. We found that one unused bit in the LPTC_STATUS PV was unset in h1omc0 and set in all the other front ends, including h1susex.

Now that Daniel has fixed the TCver error in EX, the temporary code in cds_status_ioc.py to verify the only EX error is the TCver error was removed.

DAQ Restart

Jonathan, Erik, EJ, Dave:

The DAQ was restarted primarily for the h1iopomc0 INI change due to its RCG upgrade. Several EDC changes which have been waiting for target-of-opportunity were also done.

This was a messy DAQ restart. Adding h1omc0 to the custom RCG boot server h1vmboot0 inadvertently changed the DAQ FE list to only include two frontends; h1susex and h1omc0. When the 0-leg was started initially DC0 looked good, but then we noticed that on the CDS overview only the DC0 EPICS channels for omc0 and susex were connecting, and Jonathan found GDS0 had lost a lot of channels. Around this time I also restarted the EDC on h1susauxb123 and also found that the expected number of channels was down by about 1000. At this time FW0 was writing tiny frames (only two frontends).

DC0s DAQ configuration was fixed and the 0-leg was restarted. At this point the EDC was out of sync, so it was restarted. This then generated a new H1EDC.ini which necessitated a third and final 0-leg restart (plus another GDS0).

The 1-leg restart was a single restart, with a GDS1 restart also needed.

Change of RCG reporting color on CDS Overview

Dave:

We now have two front end systems in production which are running with a custom version of the RCG. To highlight this, I have changed the non-standard-RCG indicator color from purple to black on the CDS Overview. Please see attached.

[Erik v.R., Dave, Jonathan]

h1omc0 was upgraded to support alternative duotone frequencies.

A new parameter was added to the iop model, h1iopomc0, 'duotone_frequency', currently set to '960', which configures the duotone frequency to 960/961 Hz, the standard frequencies.

The h1omc0 front end was moved from a the primary bootserver, h1vmboot1, to the secondary bootserver, h1vmboot0 using changes to the puppet configuration.

All models for h1omc0 were built and installed from RCG source rather than packages.  The git commit of the RCG source is tagged as 'h1_2024_12_10_variable_duotone'.

The omc0 front end and io chassis were shut down.  The timing card SN S2101117 was replaced with SN S2101084, which had newer firmware version 1589 that supports alternative duotone frequencies.

The IO chassis and front-end were restarted.

A few problems were encountered:

Running puppet on the secondary bootserver overwrote the DAQ master template and some EDC INI files.   The master file problem was revealed in the DAQ overview where most of the front ends were reporting no channels.  The EDC overwrite showed up as too few total EDC channels.

Running the puppet configuration on the production server re-enabled DHCP for hosts that were supposed to boot from the secondary bootserver.  These had to be manually commented out from the DHCPD configuration, then the service restarted.

To change the duotone to 1920 Hz, edit the h1ipomc0 model, set 'duotone_frequency=1920' in the parameter block, save the model.  Build and install the model then restart all models.

As part of WP 12239 we moved h0epics, cdsvmscript1, epics-burt, autoburt off of the cds0proxmox hypervisor.  This resulted in a few minutes of down time for the dust monitor IOC around 8:19am localtime.

After this we done we were able to look at cdsproxmox.  Its boot drive had failed.  After some help from Fil we got the drives replaced and reinstalled proxmox ve on cdsproxmox.  We adjusted DNS and renamed the box cdsproxmox0.

Some notes:

 * This is now in a temporary state.  We aim to retire this hardware by or at the end of O4.  New hypervisor computers are being procured.  As such we did not provision much storage on this.  Just enough to run the hypervisor, relying on the shared storage layer to handle the VM.

 * As per the proxmox administrators guild we removed cdsproxmox from the cluster prior to attaching it back as cdsproxmox0.

Closes FAMIS#28383, last checked 81688

Once again, the coherence for ITMX bias drive bias off is below the coherence threshold - this time the coherence is 0.01, much lower than the threshold of 0.1, so there are once again no new analyzed measurements for ITMX.

ETMX, ETMY, ITMX, ITMY

I've used the pico to center IM4 trans QPD.   See alogs about this history: 80604 78962 78943 78856

 After difficulty with input alignment shifts in the spring, we centered IM4 with the hopes that we could use that as a reference for any future input alignment shifts to reset the IMs.  However this didn't work well when the PMC was swapped.  Now we are recentering IM4 trans to get this new reference since we've been operating here for a while.

TJ, Camilla. WP12162. Continuing work in 76030

We installed a CFCS11-A adjustable SMA fiber collimator on the 50um fiber for the ETMX HWS fiber coupled LED source M530F2. This improved the size of the beam but it was still too large at the edge of the collimators range.

We removed HWS-L3 (D1800270) which made the beam a better size but still a little large ~ 10-15mm diameter at the periscope. We couldn't see a change in the beam by adjusting HWS-L2 (on translation stage). We aligned the system using the retrofection of the ALS beam: aligned ALS beam to output coup;er with HWS-MS1 and then adjusted the fiber collimator (mounted in mirror mount) to aligned the HWS output to the final iris. Confirmed we were getting a reflection of ETMX by mis-aligning it ~20urad. Note that the beam looks cleaner than usual photo, probably because the GV is closed so we are getting no retroflection of ITMX. Set frequecy to 1Hz. Replaced the mask photo and started the HWS code with new references.

The ETMX HWS 520nm beam in now injected into the vacuum where it hasn't been for ~months. Tagging DetChar. 

To do: measure the beam profile of beam out of fiber and re- calculate an imaging solution of the ETMX HR surface. 

GV5 and GV7 were soft closed at ~10am local time today to facilitate equipment craning for crane inspections.  They were opened at ~11am local time at the completion of the vertex crane inspection.

This morning, Ryan let me know that he got a "Check PSL chiller" verbal alarm.  Upon checking the chiller I saw the water level was a little low but not at minimum, but the usual oscillations in the water level as the chiller does its thing were getting a little close; this is likely what caused the alarm.  I added 175 mL of water to the chiller to bring the water level back up to the MAX line.

Closes FAMIS27804

CO2X was found at 29.5, i added 200ml to get it to right under the MAX line at 30.3.

CO2Y was found at 8, I added 250ml to get it to just under the MAX line at 9.4.

Early Monday morning I was alerted to the lag compressor alarm on the instrument air compressor. Both compressors were running and the tank pressure was increasing until it reached cut-out pressure. I inspected for any signs of an air leak but didn't find anything obvious. I observed the compressor and found that when the air dryer swapped desiccant towers it began blowing down a lot of air. The poly-tube connections beneath each dryer had a build-up of ice. I cleared the ice out as best as I could and observed the dryer for another several cycles, but the problem did not reoccur. We will continue to monitor the performance of the air dryer. 

Camilla C, TJ S

This morning we had another period where our range was fluctuating almost 40Mpc, previously seen on Dec 1 (alog81587) and further back in May (alog78089). Camilla and decided to turn off both TCS CO2s for a short period just to completely rule them out, since previously there was correlation with these range dips and a TCS ISS channel. We saw no positive change in DARM during this short duration test, but we didn't want to go too long and lose lock. CO2s were requested to have no output power from 16:12:30-16:14:30UTC

The past times that we have seen this range loss, the H1:TCS-ITMX_CO2_ISS_CTRL2_OUT_DQ channel and the H1:SQZ-FC_LSC_DOF2_OUT_DQ channel had noise that correlated to the loss, but theISS channel showed nothing different this time (attachment 2). We also were in a state of no squeezing at the time. So it's possible that this is a completely different type of range loss.

DetChar, could we run Lasso or check on HVETO for a period during the morning lock with our noisy range?

Ansel reported that a peak in DARM that interfered with the sensitivity of the Crab pulsar followed a similar time frequency path as a peak in the beam splitter microphone signal. I found that this was also the case on a shorter time scale and took advantage of the long down times last weekend to use  a movable microphone to find the source of the peak. Microphone signals don’t usually show coherence with DARM even when they are causing noise, probably because the coherence length of the sound is smaller than the spacing between the coupling sites and the microphones, hence the importance of precise time-frequency paths.

Figure 1 shows DARM and the problematic peak in microphone signals. The second page of Figure 1 shows the portable microphone signal at a location by the staging building and a location near the TCS chillers. I used accelerometers to confirm the microphone identification of the TCS chillers, and to distinguish between the two chillers (Figure 2).

I was surprised that the acoustic signal was so strong that I could see it at the staging building - when I found the signal outside, I assumed it was coming from some external HVAC component and spent quite a bit of time searching outside. I think that this may be because the suspended mezzanine (see photos on second page of Figure 2) acts as a sort of soundboard, helping couple the chiller vibrations to the air. 

Any direct vibrational coupling can be solved by vibrationally isolating the chillers. This may even help with acoustic coupling if the soundboard theory is correct. We might try this first. However, the safest solution is to either try to change the load to move the peaks to a different frequency, or put the chillers on vibration isolation in the hallway of the cinder-block HVAC housing so that the stiff room blocks the low-frequency sound. 

Reducing the coupling is another mitigation route. Vibrational coupling has apparently increased, so I think we should check jitter coupling at the DCPDs in case recent damage has made them more sensitive to beam spot position.

For next generation detectors, it might be a good idea to make the mechanical room of cinder blocks or equivalent to reduce acoustic coupling of the low frequency sources.