TITLE: 09/13 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Lock Acquisition
INCOMING OPERATOR: Austin
SHIFT SUMMARY:

IFO is LOCKING and is at LASER_NOISE_SUPPRESSION as of submission (07:21 UTC)

Lock Acquisition (not smooth at all)

While attempting lock acquisition, DRMI would not lock, PRMI woul not lock and none of the POP18 abd POP90 traces were anywgere near where they were supposed to be (both just showing noise about 0 counts). Guardian went to check MICH fringes about 5 times before I decided to do an initial alignment. I attemped to manually request/pause at ACQUIRE_PRMI to play the whole BS beam movement game but I couldn't see a second beam after some movement in both yaw and pitch (and guardian kept beating me to it by going into check MICH fringes). There were some strange interference patterns in the AS AIR camera that I hadn't seen before while I was attempting this.

I was checking the troubleshooting documentation throughout this, but found no relevant details to assist with this problem.

Initial Alignment (6:10 UTC)

I had issues getting guardian to perform an automatic (ish) initial alignment. It kept getting stuck trying to lock ALS when ALS had been locked steadily for a few minutes. I manually offloaded green arms and went into the next stage of initial alignment, where it started behaving and moving automatically. During initial alignment, I saw both beams overlaying in the AS AIR camera for the first time since lockloss so that is a good sign (I think). Initial alignment successfully finished so now onto locking!

Post Initial Alignment Lock (6:38 UTC):

DRMI has finally locked. This was definitely some weird alignment issue that got fixed because this time, we didn't even go through PRMI (or MICH Fringes). I expect guardian to manage locking manually.

What was the issue?

I do not know what the root of the lockloss or the lock acquisition was. As said in alog 72852 the wind had died down and there were no significant earthquakes. This leaves the BLRMs glitch lockloss coindcidence.

LOG:

Random lockloss - wind and seismic acitivity look okay at time of lockloss.

DRMI unlocked as soon as a glitch (maybe 10th one of the night) showed up on BLRMs screen. This is similar to my last few evening locklosses.

IFO is in NLN and OBSERVING as of 21:00 UTC (Sep 12) - 6hr and 35 minute lock.

Nothing else to report.

Weekly (missed week 9/4-9/8) In-Lock SUS Charge Measurements (also includes more recent 9/12 measurements).

Closes FAMIS 26056

Weekly In-Lock SUS Charge Measurements.

Closes FAMIS 26057

This leak appears to be an effect of the overpressure event of the cooling lines at Y-End, reported here, and it makes sense as to why the loss of glycol reported there too.  I was notified by Richard that someone reported a leak at Y-End, headed there and found the leak, probably had leaked for days.  The leak is located on the copper return line, probably a busted solder joint.  To stop the leak, I closed the supply and return valves, see photo, they'll remain closed until the leak is fixed.  Jordan and I let some pigs loose on the spill, they'll be picked on the next couple of days.

Unfortunately, now we have to check the condition of the hepta pump, since the pressure on the cooling lines went well above 100 psi, nominal operation conditions for such pump is a maximum of 88 psi.

I have written a python script which generates an MEDM showing the four BSC temperature sensors and the settings used to convert voltage to DegF (and DegC).

Please reference DCC-T0900287 for details.

Fil measured BSC1's skin temperature to be 69.0 - 69.5 F using the hand held remote temp sensor.

BSC1 initially came back with a low and noisy voltage. Richard reattached the cabling which fixed the issue.

To open this MEDM, from a recent SITEMAP, it is the last entry in the FMCS pull down (the PEM pull down is totally full)

TITLE: 09/12 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 146Mpc
OUTGOING OPERATOR: TJ
CURRENT ENVIRONMENT:
    SEI_ENV state: CALM
    Wind: 10mph Gusts, 7mph 5min avg
    Primary useism: 0.02 μm/s
    Secondary useism: 0.11 μm/s
QUICK SUMMARY:

IFO is in NLN and Observing as of 21:00 UTC

TITLE: 09/12 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 148Mpc
INCOMING OPERATOR: Ibrahim
SHIFT SUMMARY: Maintenance day recovery was fairly easy considering the h1iopsusb123 went down and tripped the associated sei platforms. Automated initial alignment and lock. We've been observing for about 2.5 hours now.
LOG:

• 1515 Lock loss
• 2054 Observing
• 2059-2100 Briefly out of observing for TCSX relock

Recovery of h1susb123

TJ, Fil, Erik, Jonathan, Dave:

During the CER in-rack work a timing glitch stopped all the models on h1susb123. After the rack work was completed, we restarted the models by soft booting h1susb123 (sudo systemctl reboot).

When the computer came back we noted that the first IO Chassis Adnaco backplane (the one with the timing card on it) was not visible from the computer, but the other 3 were.

We did a second restart, this time Erik power cycled the IO Chassis and I powered down h1susb123. This time the system came back up with a full compliment of cards.

WP11414 Picket Fence Update

Erik, Dave:

New picket fence server code was started on nuc5, along with a new IOC on cdsioc0. Two new GPS channels were added which record the time the server was started, and the last time picket fence obtained and processed its seismic data. DAQ + EDC restart was required

The new channels are  H1:SEI-USGS_SERVER_GPS and H1:SEI-USGS_SERVER_START_GPS

WP11423 h1calcs bug fix

Jeff, Dave:

A new h1calcs model was installed, no DAQ restart was needed. Please see Jeff's alog for details.

WP11420 Add calibrated PEM BSC temperature channels to DAQ

Richard, Fil, Robert, Dave:

New h1pemcs, h1pemex and h1pemey models were installed. The BSC temperature raw_adc_counts are now converted to DegF using DCC-T0900287 settings. The raw counts DAQ channel keeps it name, for each BSC a new DEGF and DEGC fast 256Hz channel was added to the DAQ, along side equivalent MON slow channels. DAQ restart was required.

Reboot of FMCS IOC

Patrick, Jonathan, Dave:

The FMCS IOC computer was rebooted to verify it starts ok following the last week's FMCS activity. I recovered the alarm settings.

DAQ Restart

Jonathan, Dave:

The DAQ was restarted for the above PEM and Picket Fence changes. Other than a second restart of gds1 being needed, this was an unremarkable restart.

The turbo pump stations at X-End and X-Mid station were taken thru their functionality test, no issues noted with both systems.

EX
Scroll pump hours: 6311.9
Turbo pump hours: 263.4
Crash bearings life: 100%

MX
Scroll pump hours: 204.9
Turbo pump hours: 111.4
Crash bearings life: 100%

FAMIS tasks 24843 and 24867

The pumping station is located in the Mechanical Room, the vacuum line in question starts there at a temporary pumping station, then penetrates through the wall, into the LVEA, approximately between HAM5 and HAM6. Then it joins the direction of the main tubes, turns north, and then turns east at BSC2, and runs a little bit over BSC7. The purpose of this noise examination is to justify if we can pump this line during observation. It would be quite advantageous, as this is crucial for the cleaning attempt of this line, after the Hepta incident.
On this pumping cart we have a Turbo and a backing pump. The backing pump definitely the noisier, it runs at 1660 rpm = 27.67 Hz. The Turbo pump runs at 60000 rpm = 1000 Hz.

During the measurement, the pumping cart was switched on and off, and the changes in the operation of the pumps have been noted. Beforehand, a loud noise was generated on the tubing, by hitting it with a wrench. The noise measurement, in time:
13:02 - gate valve was closed - some minimal noise: the pumping cart was on at the beginning of the measurement
13:04 - 13:05: hitting the tube with a wrench, approx 2 Hz: quite some noise
13:05 - gate valve reopened - some minimal noise (the reason behind the opening/closing of the gate valve is non-related to the noise-experiment, it is only VAC-related)

13:06:42 - press STOP - this has nothing to do with the noise
13:10:20 - backing pump stops: 27.67 Hz noise-source disappeared. Also, Turbo pump's freq was changed to 850 Hz
13:11:50 - Turbo pump's freq. was changed to 775 Hz
13:13:05 - Turbo pump's freq. was changed to 700 Hz
13:13:20 - Press ON: backing pump starts - 27.67 Hz noise source reappears
13:15:05 - Turbo pump is at full speed: 1000 Hz

13:18:00 - press STOP - this has nothing to do with the noise (1E-4 Torr)
13:21:38 - backing pump stops: 27.67 Hz noise-source disappeared. Also, Turbo pump's freq was changed to 850 Hz
13:23:00 - Turbo pump's freq. was changed to 775 Hz
13:24:15 - Turbo pump's freq. was changed to 700 Hz
13:25:00 - Press ON: backing pump starts - 27.67 Hz noise source reappears
13:26:50 - Turbo pump is at full speed: 1000 Hz

13:29:55 - press STOP - this has nothing to do with the noise (1E-4 Torr)
13:33:30 - backing pump stops: 27.67 Hz noise-source disappeared. Also, Turbo pump's freq was changed to 850 Hz
13:34:55 - Turbo pump's freq. was changed to 775 Hz
13:36:10 - Turbo pump's freq. was changed to 700 Hz

13:36:35 - gate valve was closed (8.5E-2 Torr)

Maintenace recovered.

WP 11418
ECR E2300035
FRS 26604

Monitor boards D1900052 were installed in BS and SRM M2/M3 in April (alog 68796).
Today the D1900052 monitor boards were removed and original monitor boards D070480 reinstalled.

Beam Splitter (Rack SUS-C5, U26):
Chassis Serial Number S1100039
Old Monitor Board MON156 - Installed
New Monitor Board S2100109 - Removed

SRM M2 (Rack SUS-C7, U26):
Chassis Serial Number S1100035
Old Monitor Board MON177 - Installed
New Monitor Board S2100483 - Removed

SRM M3 (Rack SUS-C7, U22)
Chassis Serial Number S1000356
Old Monitor Board MON128 - Installed
New Monitor Board S2100108 – Removed

The D1900052 boards that were removed from BS/SRM were installed in PRM M2/M3 and PR3 M3.

PRM M2 (Rack SUS-C4, U19):
Chassis Serial Number S1100015
Old Monitor Board MON222 - Removed
New Monitor Board S2100483 - Installed

PRM M3 (Rack SUS-C4, U18):
Chassis Serial Number S1100025
Old Monitor Board MON233 - Removed
New Monitor Board S2100109 - Installed

PR3 M3 (Rack SUS-C4, U16)
Chassis Serial Number S1100047
Old Monitor Board MON223 - Removed
New Monitor Board S2100108 – Installed

The LVEA has been swept, nothing abnormal to report - all checklist items have been met.

J. Oberling, A. Jennings

With a relatively light maintenance window and a chance for some extended Laser Hazard time, we finally completed the installation of an old OpLev ECR, FRS 4544.  Austin and I removed the existing 10m single-mode fiber and installed a new, 3m armored single-mode fiber, and placed the laser in a "thermal isolation enclosure" (i.e. a Coleman cooler).

To start, I confirmed that there was power available for the new setup; the OpLev lasers are powered via a DC power supply in the CER.  I read no voltage at the cable near the ITMx OpLev, so with Fil's help we traced the cable to its other end, found it unplugged, and plugged it in.  I confirmed we had the expected voltage, which we did, so we moved on with the installation.  We had to wait for the ITMx front end computer to come back up (had tripped as part of other work), so while we waited Austin completed the transition to Laser Hazard.  We took a picture (1st attachment) of the ITMx OpLev data (SUM counts, pitch and yaw readings), then powered down the laser.  We placed the laser in the cooler and plugged in the new power supply; laser turned on as expected.  We then installed a Lexan viewport cover and removed the cover from the ITMx OpLev transmitter pylon.  The old 10m fiber was removed, and we found 2 areas where the fiber had been crushed due to over-zealous strain relief with cable ties (attachments 2-4; this is why we originally moved to armored fibers); I'm honestly somewhat surprised any light was being passed through this fiber.  We installed the armored fiber, being careful not to touch the green camera housing and to not overly bend the fiber or jostle the transmitter assembly, and turned on the laser.  Unfortunately we had very little signal (~1k SUM counts) at the receiver, and the pitch and yaw readings were pretty different.  We very briefly removed the Lexan cover (pulled it out just enough to clear the beam) and the SUM counts jumped up to ~7k; we then put the Lexan back in place; we also tried increasing the laser power, but saw no change in SUM counts (laser already maxed out).  This was an indication that we did manage to change the transmitter alignment during the fiber swap, even though we were careful not to jostle anything (it can happen, and it did), and that the Lexan cover greatly changes the beam alignment.  So we loosened the locking screws for the pitch and yaw adjustments and very carefully adjusted the pitch and yaw of the launcher to increase the SUM counts (which also had the effect of centering the beam on the receiver).  The most we could get was ~8.3k SUM counts with the Lexan briefly removed, which then dropped to ~7k once we re-installed the transmitter cover and completely removed the Lexan (no viewport exposure with the transmitter cover re-installed).  We made sure not to bump anything when re-installing the transmitter cover, yet the SUM counts dropped and the alignment changed (the pitch/yaw readings changed, mostly yaw by ~10 µrad).  Maybe this pylon is little more loose than the others?  That's a WAG, as the pylon seems pretty secure.

I can't explain why the SUM counts are so much lower; could be the difference between the new and old fibers, we could have really changed the alignment so we're now catching a ghost beam (but I doubt this, we barely moved anything).  Honestly I'm a little stumped.  Given more time on a future maintenance day we could remove the receiver cover and check the beam at the QPD, but as of now we have what appears to be a good signal that responds to pitch and yaw alignment changes, so we moved on.  We re-centered the receiver QPD, and now have the readings shown in the 5th attachment; ITMx did not move, it stayed in its Aligned state the entire time.  This is all the result of our work on the OpLev.  We'll keep an eye on this OpLev over the coming days, especially watching the SUM counts and pitch/yaw readings (looking for drift and making sure the laser is happy in its new home; it is the oldest installed OpLev laser at the moment).  The last few attachments are pictures of the new cooler and the fiber run into the transmitter assembly.  This completes LHO WP 11422 and FRS 4544.

J. Kissel, D. Barker
WP #11423

Dave has graciously compiled, installed and restarted the h1calcs model. In doing so, that brings in the bug fix from LHO:72820, which fixes the issue that the front-end, CAL-CS KAPPA_UIM library block was receiving the KAPPA_TST uncertainty identified in LHO:72819.

Thus h1calcs is now using rev 26218 of the library part /opt/rtcds/userapps/release/cal/common/models/CAL_CS_MASTER.mdl.

I'll confirm that the UIM uncertainty is the *right* uncertainty during the next nominal low noise stretch later today (2023-09-12 ~20:00 UTC).