Jonathan, Erik, Dave, Patrick As part of WP12358 we installed two new digivideo server h1digivideo5 and h1digivideo6. These were racked in the aux rack. To make room we removed two old conlog servers. Both servers were installed with Debian 12 and are under puppet configuration and running the 0.1.16 version of the pylon-camera-server software. They are connected to the new video1 switch. We migrated cameras off of h1digivideo0, h1digivideo1, and h1digivideo3. These machines have been powered off. h1digivideo4 is running the following cameras H1-VID-CAM-ALS-X H1-VID-CAM-ALS-Y H1-VID-CAM-FC-REFL H1-VID-CAM-MC-REFL H1-VID-CAM-MC-TRANS H1-VID-CAM-MC1 H1-VID-CAM-MC2 H1-VID-CAM-MC3 H1-VID-CAM-PR2 H1-VID-CAM-PR3 H1-VID-CAM-PRM H1-VID-CAM-ISCT6-AS-AIR H1-VID-CAM-SRM H1-VID-CAM-SQZT7-OPO-GR-TRANS H1-VID-CAM-SQZT7-IR-TRANS h1digivideo5 is running the following cameras H1-VID-CAM-ETMX H1-VID-CAM-ETMY H1-VID-CAM-FC1 H1-VID-CAM-FC2 H1-VID-CAM-FCES-GREEN H1-VID-CAM-FCES-IR-TRANS-B H1-VID-CAM-SHG-TRANS H1-VID-CAM15 We did a daqd restart to remove monitoring information regarding h1digivideo3 and to reflect changes in EPICS variables due to migrating from older versions of the camera software. We still need to migrate cameras off of h1digivideo2. That will likely happen next Tuesday.
FAMIS27777
I didn't find any major issues with the pipes themselves, no leaks were found and the supports that were added last year (alog76528) seem to be helping. The flex joints still seem good and the plastic wrap around it also look OK. The same issues reported in the last inspection are still present (alog80015): cuts/scratches in pipes and a blocked outlet under HAM4 that also stressed the pipes there a bit from people pushing them out of the way to use the outlet. After getting Richard's approval, I moved the outlet up on its support beam and on a conduit support further down to clear the TCS pipes and relieved some of the stress on the pipes there - Pic1 Pic2
TITLE: 03/04 Day Shift: 1530-0030 UTC (0730-1630 PST), all times posted in UTC
STATE of H1: Preventive Maintenance
INCOMING OPERATOR: Ibrahim
SHIFT SUMMARY: DRMI/PRMI has been giving us issues in the maintenance recovery, PRMI_ASC doesn't seem to actually improve the signal or camera. The wind has been increasing over the past 4 hours and the 2ndary microseism has been decreasing. We've run 1 full IA and a 2nd with only SRC. Some yellow PSL laser room alarms, probably from the wind.
LOG:
Start Time | System | Name | Location | Lazer_Haz | Task | Time End |
---|---|---|---|---|---|---|
15:37 | CDS | Erik | Office | N | SUS EX restart for RCG upgrade | 16:17 |
15:56 | EE | Ken | EndY | N | Mech room compressor work | 21:40 |
16:04 | FAC | Tyler & Chris | Optics lab | N | Move stuff | 17:42 |
16:10 | FAC | Tyler | Old bake lab | N | Removing old elevator, NOISY AND DIRTY | 17:42 |
16:11 | LSC | Daniel | LVEA | N | REFL measurements | 18:41 |
16:11 | FAC | Kim & Nellie | EndY | N | Tech clean | 17:11 |
16:12 | CDS | Jonathan | Office | N | Digivideo work | 16:39 |
16:17 | CDS | Erik | CER | N | Working on broken WAPs (LVEA) | 19:45 |
16:22 | EE | Fil | LVEA | N | LY vac rack ups | 16:38 |
16:24 | IAS | Jason, RyanC | LVEA | N | FARO surveying | 20:15 |
16:26 | VAC | Jordan, Travis | EY MER | N | Mech room, compressor | 21:54 |
16:39 | SEI | Fil | CER | N | SEI rack electronics | 19:17 |
16:50 | AOS | Richard | LVEA | N | Checks | 17:07 |
16:53 | FAC | Mitchell, Jim, Betsy | LVEA | N | Checks around HAM1 | 19:44 |
16:58 | CAL | Francisco | EX | YES | PCal measurement | 18:12 |
17:03 | VAC | Janos, Gerardo | EX MER | N | Compressor testing | 21:54 |
17:14 | TCS | TJ | MER/LVEA | N | Chiller piping checks | 18:03 |
17:24 | FAC | Chris (+Pest Control) | LVEA, Arms, Ends | YES (EX) | FAMIS tasks, Pest control | 18:30 |
18:06 | FAC | Tyler | MX, MY | N | 3IFO checks | 19:06 |
18:17 | TCS | TJ | LVEA | N | Adjusting TCS lines | 18:32 |
18:20 | FAC | Nellie | LVEA | N | Technical cleaning | 19:34 |
18:25 | CAL | Jeff | CER/LVEA | N | Pictures at SUS racks | 19:43 |
18:31 | FAC | Chris | MER | n | Tumbleweed clearing | 21:23 |
19:14 | FAC | Tyler | All Bldgs | N | Measuring technical slabs | 21:14 |
19:15 | CDS | Erik | EY | N | Plugging in timing system | 20:10 |
19:15 | FAC | Richard | EY | N | Checking in w/ Ken | 19:48 |
19:17 | EE | Fil | LVEA | N | Table electronics documentation | 20:13 |
19:53 | AOS | Betsy, Mitchell | LVEA | N | HAM1 Checks | 20:13 |
21:20 | ALS | TJ, Sheila | LVEA, ISCT1 | LOCAL | Table adjustment | 22:20 |
21:27 | OPS | RyanC | LVEA | N | Sweep | 21:50 |
21:39 | ISC | Siva, Mayank | Opt Lab | N | ISS array | Ongoing |
21:53 | VAC | Gerardo | Optics lab/ VAC prep | N | Find a viewport | 21:59 |
22:02 | ISC | Keita | Optics lab | N | Join ISS work | Ongoing |
23:22 | VAC | Janos | EndX | N | Quick measurement | 23:36 |
Some of the maintance items completed today includes:
TITLE: 03/04 Eve Shift: 0030-0600 UTC (1630-2200 PST), all times posted in UTC
STATE of H1: Preventive Maintenance
OUTGOING OPERATOR: Ryan C
CURRENT ENVIRONMENT:
SEI_ENV state: EARTHQUAKE
Wind: 28mph Gusts, 16mph 3min avg
Primary useism: 0.07 μm/s
Secondary useism: 0.33 μm/s
QUICK SUMMARY:
IFO is LOCKING and having issues recovering from maintenance with the following being reported from DAY Ops (Ryan C):
Sheila and I took a bit of time at the end of maintenance to touch up the COMM and DIFF paths on ISCT1.
We first removed one of the shims that was under the DCPD circled in light blue. This was shimmed up a few weeks ago (alog82783) to get more light on the pd without realigning the entire path since we had more PR suspension moves to do. Two shims were added at the time, and we were only able to remove the larger of the two, but it placed the beam close to center in height. The second attachment shows before the shim was removed.
The COMM path looked clearly not well coaligned viewed from before the last lens in the path. Sheila walked the two optics circled in pink to bring them closer together and ended up with a positive beatnote. At first we could increase the beatnote but visually the two beams seemed to get worse, but she ended up getting to a spot that worked for both.
The DIFF beatnote was already pretty good, but we touched it up a bit more using the optics circled in red. While here, we also added a beam dump behind the lower of the two circled optics. We ended up with a DIFF beatnote around -8.
Since PI31 has been wanting to sit higher (83142) and has been regularly ringing up following the EY ring heater change (82962), I've lowered the threshold for turning the damping on on PI31. It previously would turn on once PI31 RMS went above 4, but I've lowered it to 3. This means starting about 4 hours into our locks, the damping on EY will be on almost continuously, but that's okay. This will hopefully stop the sudden ringups from happening every 30-45 minutes.
J. Kissel, D. Sigg ECR E1700228 WP 12370 The last piece of today's puzzle -- after understanding that PR3's optical lever (oplev) whitening chassis is NOT in newly labeled SUS-R1 (see LHO:83156), but in SUS-R2 (see LHO:83165, and when I first realized this in LHO:63849) -- I can now comfortably identify the physical cables that Daniel "moved" in future SUS HAM 1 2 wiring diagram to make room for PM1. In short: We need to move the D9F end of cable "H1:OP LEV_PR3_AA" or "H1:SUS-HAM2_88" - from SUS-C4 U11 AA chassis spigot "IN25-28," which is CH24-27 via ADC0 in/on the sush2b IO chassis / front-end computer (and shipped over to PR3 via IPC in the top level of h1susim.mdl) - to SUS-C4 U32 AA chassis spigot "IN29-31," which shall be CH28-31 via ADC1 in/on the sush2a IO chassis / front-end computer (and be added to the top level of h1suspr3.mdl as a direct read of ADC1) and then make model changes accordingly. Pictured are - Front view of SUS-R2 U12 ("U31" in iLIGO rack terms, as its labeled) oplev whitening chassis showing that this is indeed the PR3 optical lever, as stated in LHO:63849. - Back view of SUS-R2 U12 oplev whitening chassis showing the field end of the SUS-R2 to SUS-C4 cable, labeled "H1:OP LEV_PR3_AA" or "H1:SUS-HAM2_88" - SUS-C4 U11 AA chassis where the remote end of the cable is connected to "IN25-28" - SUS-C4 U32 AA chassis were we want to use "IN29-32." Why was this so hard? The history of drawing optical levers and SUS in HAM1 in wiring diagrams is sordid. (1) The PR3 optical lever was originally drawn in the SUS HAM 2 wiring diagram in the aLIGO era. See page 26 of D0902810-v7. However, in these aLIGO era drawings, field racks weren't defined so each site put their optical lever whitening chassis where-ever, and connected them up in a way that made sense to them. Complicating the allocation were the never-used, now-decommissioned HAM-ISI optical levers, and as drawn in D0902810-v7, the HAM2 ISI optical lever was to pair with this PR3 optical lever. (2) In the effort to, among other things, define field racks for the first time, Luis and I circa early 2023 (see 5097), we cleaned up the PR3 optical lever chain, arguably in correctly, and included PR3 and SR3 in the SUS HAM 3 4 wiring diagram D1000599-v8, see pages 5 and 7 (PR3 ADC) or 6 (SR3), respectively. (3) The "missing" PR3 optical lever continued when we, as a result of the plan to integrate SUS in O5 (G2301306), created a new BSC 2 HAM 3 4 wiring diagram, D2300383, and in doing so we left out the PR3 optical lever chain. (See discussion below about the SR3 optical lever chain). In parallel, (4) as the HAM 2 wiring diagram, D0902810, became a very quick-and-dirty D0902810-v8 circa 2017 in order to find space for the then O3 SQZ system in HAM6 -- see, e.g. LHO:38883, or other alogs where the "h1sushtts" model changes are mentioned. (5) Luis and I then, circa 2023 as the last field racks to draw, converted the drawing to the "modern" wiring diagram with D0902810-v9. Since it included the RMs, but no change for O5, we renamed it the HAM 1 2 wiring diagram. Again, because we'd put PR3 optical lever in D1000599-v8, nothing was shown in D0902810-v9. So, when Daniel comes along, perusing for spare channels, he implicitly makes the recommendation to move the PR3's AA / ADC input from sush2b to sush2a, looking to make room for the incoming PM1, and further JM1 - 3. So now it appears back again in D0902810-10, on page 10, going into the U32 AA Chassis "IN29-32" spigot! This is "necessary" and "natural," because after this change, - everything on sush2a is related to the triples in HAM2 - PR3 optical lever can be directed read out from ADC1 of sush2a, rather than having to receive it via IPC from another IO chassis (sush2b) - sush2b drives only single suspensions, even though they're split between HAM2 (the IMs) and HAM3 (RMs, PM1, and the future JMs). - that leaves us open and capable, eventually, if we mass produce LIGO 32-channel DACs and we can claw back lots of 8 channel card slots, perhaps sush2a and sush2b can naturally evolve to simply sush1 and sush2, much like sush56 is to evolve to sush5 and sush6. So -- a pretty dang simple change, but it was just tough to follow with the wiring diagrams alone. FUTURE DRAWING ACTIONS: (I) Need to update HAM 1 2 wiring D0902810-10 to include the a page for the "sensor and field" wiring for PR3 that's currently in D1000599-v8 page 5, so that the ADC input (on current page 10) doesn't appear out of "thin air" in D0902810-10. (II) (For now, we have to leave SR3 on sush34, since there're no spare ADC spigots in the sush56 ADCs' AA chassis, see page 6 of D1002740-v10) (III) When we split the sush56 computer into two, sush5 and sush6, for O5, in the sush5 drawing, currently D2300380-v3, we should include the a page for the "sensor and field" wiring for SR3 that's currently in D1000599-v8 page 5. In doing do, we can move the AA chassis end of SUS_OPLEV_SR3_AA :: from SUS-C2 sush34 HAM 34 IO chassis's AA U33 of SUS-C1 "IN17-20" spigot :: to the SUS-C8 sush5 HAM 5 IO chassis's AA U33 of SUS-C7 "IN17-20" spigot that's currently spare. There there'll be some semblance of symmetry and "final home" for SR3 optical lever. (IV) Need to update D1000599-v8 to remove all evidence of PR3 and SR3. PR3 will "already" be represented in the HAM 1 2 wiring diagram D0902810, and SR3 will be represented in the HAM 5 wiring diagram D2300380, and the HAM 3 4 wiring diagram D1000599 will be deprecated anyways in favor of the BSC2 HAM 3 4 wiring diagram, D2300383, which "correctly" only includes the Beam Splitter optical lever. BUT FOR NEXT WEEK: All we need to do is the above summarized single cable move -- and make the corresponding changes to the h1sushtts and h1suspr3 front-end models.
Picket Fence wall display was updated. The update changes two of the source station channel definitions:
"US_MSO:00BHZ" changed to "US_MSO:00HHZ"
And
"IU_TEIG:00HHZ" changed to "IU_TEIG:00BHZ" (only used at LLO)
J. Kissel, F. Clara ECR E1700123 More yak shaving, but at the end of today's kLog series you'll understand how to execute Daniel's implicit plan in D0902810-v10 for moving cables around to make room for an incoming tip-tilt, aka HTTS, to eventually be named "PM1," i.e. Pop wfs steering Mirror 1. In this kLog: There're 2x cables DB9 available for the cable run from SUS-R1 to SUS-C4 to cover the needs of that run for PM1, we may not need to buy new cables. In 2022, I inventoried all optical lever whitening chasses in LHO:63844. In that aLOG I confirmed that the never-existed / now-officially decommissioned HAM2 and HAM3 ISI optical levers were disconnected from their respective optical lever piers. Today, while poking around the racks, I identified that - The last version of a wiring diagram that had any record of these HAM ISI optical levers was :: HAM3 D1000599-v7 (it's now at -v8, and in O5 D1000599 will be superceded by D2300383). :: HAM2 D0902810-v7 (it's not at -v10). - These diagrams disagree with how H1 started cable up the ISI optical levers, which have both the HAM3 and HAM2 ISI optical levers routed to the Oplev Whitening Chassis in U9 (ish) of SUS-R1. :: According to the above -v7 drawings, the intent was to have the HAM3 ISI optical lever readout by the Oplev whitening chassis in SUS-R2 over by HAM3. - The wiring diagrams specified the 2x long DB9 cable run between the whitening chassis (in the field rack) and the AA chassis (in the CER) to be called :: (IFO):OPLEV-HAM2_AA :: (IFO):OPLEV-HAM3_AA - As H1 labeled them, they're also (respectively) :: H1:SUS_HAM2_78 :: H1:SUS_HAM3_161 - These HAM2 and HAM3 Whitening-to-AA cables *are pulled* from SUS-R1 to SUS-C4 and SUS-C1, respectively, but they're dangling unconnected to any AA. In short -- the Oplev whitening chassis S1101555 in SUS-R1 U9 *looks* full populated and in use, but every "other end" of the cables plugged into it are disconnected. This chassis, S1101555 in SUS-R1 U9 can *absolutely* be powered down and removed from the rack, pushed to our spares supply. So, interestingly to our mission, there's two D9M (at SUS-R1 end) to D9F (at CER end) available to be repurposed. Coincidentally, according to Daniel's design in D0902810-v10, PM1 needs two D9 cables... sort of. - SENSOR CABLE: The OSEM PD output of the SUS-R1 U38 Dual, 8ch satamp, D1002818 is mapped from the "PD Outputs to AA" D9F directly to the SUS-C4 U11 "IN25-28" D0902783 8x D9M input AA chassis. :: one of the above ISI oplev cables will meet this need perfectly. - DRIVE CABLE: The D1100687-v1 or v2 HAM-A coil driver, which uses the driver board D080276, sends out the four coil drive outputs on a D15F on J4. But, the 4x differential channels can absolutely fit on a DB9, as is evident by the Dual, 8ch satamp, D1002818 which accepts the Coil Drive Inputs at J1 as a D9M. In A+ circa 2021-2022, when we started regularly using this HAM-A coil driver + 8ch Dual Satamp combo, Luis drew up the cable solution to this problem, a custom D15M to D9F cable, D2200034. :: So, we "just" need to make a short version of this oft-used custom D15 to D9 cable, D2200034, and then we can use the other cable. That being said -- Daniel says he's "already ordered the cables." Ah HA! But -- I worry that the wrong cable has been ordered -- because in D0902810-v10 page 8, Daniel did the sensible thing and copied and pasted the RM1 and RM2 wiring chains from D0902810-v9 page 8 to create the PM1, and JM1-3 chains on -v10 page 9. But alas, there was a typo on the label of this DRIVE CABLE, citing that it should be D2100747, a D15M to D15F cable. It should be the D2200034 D15M to D9F mentioned above. So -- if we did end up ordering the wrong cable, here's the solution: With only one short adapter made, we can reuse these D9M to D9F cables for the OSEM PD and Coil Drive cables of PM1, if we want or need. Or anything else future that needs a long from from SUS-R1 to the SUS racks in the CER. (Future readers beware -- the plan may be to fix the typo D2100747 > D2200034 cable label typo in the D0902810-v10 drawing by keeping the same version number in the DCC but sneakily replacing the files -- so if you're reading this in 2026, the comments about the typos in -v10 may not be there...)
Some supporting pictures from the above conversation. - 2025-03-04_SUSR1_ISIHAMOplevWhitening_FrontView_U9.jpg Front view of S1101555 Oplev Whitening Chassis in SUS-R1 U9(ish). - Zoom in on HAM3 and HAM2 cable connections at the front of the chassis in SUS-R1. - Zoom in on HAM3 and HAM2 cable connections at the back of the chassis in SUS-R1. - The disconnected end of the HAM3 cable dangling near SUS-C1 in the CER - The disconnected end of the HAM2 cable dangling near SUS-C4 in the CER.
I swept the LVEA following the conclusion of most maintenance activities, the last group in will double check their area and get the lights on their way out. Nothing really of note.
WP 12366
Started installation of the required electronics (ISI Interface/AA/AA/ISI Coil Drivers/Binary Chassis) for the HAM1 ISI upgrade. IO chassis will need to be updated.
All existing electronics need to be powered down to move power cables to lower ports on the ±18V DC power strip. New electronics are installed in the upper U slots of the rack and standard power cables are not long enough. Work will be coordinated with the IO chassis upgrade.
Waiting on the in-rack and field cables to be delivered onsite.
J. Kissel, F. Clara I'm yak shaving, but at the end of today's kLog series you'll understand how to execute Daniel's implicit plan in D0902810-v10 for moving cables around to make room for an incoming tip-tilt, aka HTTS, to eventually be named "PM1," i.e. Pop wfs steering Mirror 1. In this kLog: I've labeled the U heights of SUS-R1. SUS-R1 is an old iLIGO rack that was never labelled, but repurposed in aLIGO to house (mostly) HAM2 SUS satellite amps, i.e. the triples MC1, MC3, PRM, PR3, and the singles IM1, IM2, IM3, and IM4. (It also houses a CDS network switch, a PEM test panel, and a deprecated / never used HAM2 and HAM3 ISI optical lever whitening chassis.) In aLIGO, all U-slots in U42-ish height racks were "converted" to being labeled by industry standard with the top being U42, and the bottom being U1. These iLIGO racks don't have enough slots to populate all 42 U-height slots, so the compromise is to *draw* 42 U-heights, but "never use" U1. The last bit of the story: the drawing that governs the content of these racks is D0902810. However, when the aLIGO field racks were populated with stuff, (a) The sites made a custom non-standard 3.25-ish U-height "craddle" to hold the non-rack-mountable UK satamps. (b) that drawing did not cover the layout of the field racks, so each sites kinda "did what they wanted." The issue (b) was been rectified by 2023 as Louis Sanchez and I brought D0902810 up to -v9, see IIET:5097. So, that's why the U-height labels don't awesomely line up with the chassis that are installed. See - Screenshot of depiction of the rack in page 13 D0902810-v10, with the rack labeled "top down" from U42 at the top and U41 at the bottom (even though the H1 "as-built" version of this design doesn't have a U1). - single "before" picture - The rest of the attachment: pictures of the labels.
In order to help narrow the cause of on-going but rare timing comparator discrepency with the CNS clock at EY, I installed a rubidium clock on port 4 of the same comparitor.
The clock is on the floor under the computer rack. A BNC cable runs from the clock under the roll up door, and over the racks in the CER to the comparator.
Port 7 of the comparator didn't work.
LVEA WAP is now working. When either the LVEA WAP or the CER WAP is toggled on or off, the toggled WAP must be pinged from the 'wap-control' server before it will reconnect and show up on MEDMs.
Other WAPs are working normally. This seems to be a networking issue and not a problem with the WAPs themselves.
I talked to Sheila this morning about turning on the ETMX glitch limiters from this alog, so I've edited ISC_LOCK to turn this back on. This will cause SDF diffs on ETMX when we get to NLN, but otherwise shouldn't interfere with locking. If it's necessary to turn this off, just comment out lines 5462-5464:
log('turning on esd limits to reduce ETMX glitches')
for limits in ['UL','UR','LL','LR']:
ezca.get_LIGOFilter('SUS-ETMX_L3_ESD_%s'%limits).switch_on('LIMIT')
Still unclear if this is helping, but we turned it off ~Feb 12 after running it for almost 2 weeks.
[Erik v.R., Dave]
h1susex models were re-built with the same version of the RCG as h1omc models.
git tag is 'H1_rollback_2025_01_14'
Dolphin started reporting errors while shutting down the models. We tried to reboot the front end, but it crashed, so we hard reset it.
Following T2500040-v1 the phase delays were measured for LSC REFL_A, LSC POP_A, ASC REFL_A, ASC REFL_B, and ASC POP_X. Results in the attached pdf.
After replacing the chiller pump seal, I switched the lead unit to enable to chiller to run, but it failed on water flow. After troubleshooting the chiller, I found that the wiring harness for the flow switch had been chewed on by an animal. I will see about splicing the damaged wiring back together, but until that is done we are down one chiller at End X.
This has been repaired and the chiller is now running.