travis.sadecki@LIGO.ORG - posted 14:03, Tuesday 31 October 2023 - last comment - 14:45, Tuesday 31 October 2023(73877)
MY and EY Turbo functionality test
Fil, Dave, control room:
In preparation for Fil's replacement of the +24V DC Power Supply which is powering h1lsc0 and h1asc0's IO Chassis, I have powered down these two front end computers.
Process was:
IPMI Reports:
WP 11502
Kepco power supply replaced. Both IO chassis and FE computers powered on.
Tue31Oct2023
LOC TIME HOSTNAME MODEL/REBOOT
12:59:06 h1lsc0 ***REBOOT***
12:59:07 h1asc0 ***REBOOT***
13:00:42 h1asc0 h1iopasc0
13:00:42 h1lsc0 h1ioplsc0
13:00:55 h1asc0 h1asc
13:00:55 h1lsc0 h1lsc
13:01:08 h1asc0 h1ascimc
13:01:08 h1lsc0 h1lscaux
13:01:21 h1asc0 h1ascsqzifo
13:01:21 h1lsc0 h1sqz
13:01:34 h1lsc0 h1ascsqzfc
Attached sqz sdfs that Naoki and I set so that SQZ would come back in a good (with FC DOWN) state. Lots of these will need to be re-accepted later when SQZ is relocked as some of our sfae.snap and observe.snaps are linked files.
Jenne checked sdfs for h1asc, h1ascimc, h1lsc, h1lscaux.
Attached are SDF tables for h1lsc, h1lscaux, and h1asc. There were no diffs for h1ascimc.
The tables have the mask set to ALL, so that it was showing also differences that are not monitored, since *everything* gets reverted to the safe value when we boot a computer.
For h1lsc, I only accepted the TR QPD B offsets, to keep them roughly close to how they are when we've been locking. These do get reset every lock (which is why they are not monitored), but if we don't accept them then we'll struggle more than usual with finding IR when we're ready to relock. Everything else is either guardian controlled, or otherwise not so important.
For h1lscaux, I don't really know why those values of 500 were saved to have those oscillators on, but the scope shows that we don't usually actually use them. This sdf table is not reverted upon each relock (but would have been when we rebooted), so that's likely why we never noticed / cared about these values. Anyhow, we don't want random oscillators on (even if the output matrix is zeros so those don't go anywhere), so I accepted them to zeros.
For h1asc, I didn't accept any changes, but just in case we have confusion later, I took a screenshot of the diffs that we have.
There were no diffs at all for h1ascimc, so I didn't bother taking a screenshot of an empty SDF table.
Austin is working on bringing the IMC back online, and the alignment looks quite poor. In retrospect, I should have (and forgot to do) offloaded the IMC WFS to the suspensions, since the reboot will have lost the integrated alignment offsets that we were sending to the IMC suspensions. Austin is going to revert the IMC sus to their earlier top mass osem values as a way of hand-offloading the WFS, and hopefully the IMC will lock and behave after that.
H1-ISC-C1 IO ASC & LSC power supply was replaced on Tuesday. Supply location H1-VDC-C2 slot 22A (left side).
Outgoing Supply S1201935 Incoming supply (blank) New supply has ball bearing fan.
Outgoing supply had sleeve bearing fan that failed due to being worn out. Will replace with ball bearing fan and put back into spares inventory when parts are available.
F. Mera, F. Clara, M. Pirello
Maintanence day is currently chugging along with most of the work already completed. The main items left to do are the EX PCAL measurement, VAC turbo pump/scroll pump and cross installation, and potential OMC scans. It was noted that sometime this morning, the Kepco power supply fan in the CER powering the ASC/LSC IO chassis malfunctionated and is now making a loud rattling sound (the leading theory is that one of that supply fans broke off). After consulting with commissioners and Dave/Fil, it has been decided to replace this power supply fan ASAP. The commissioning team has just completed reconciling SDFs and Dave has just powered down the IO chassis and the EE team will soon go into the CER to swap the power supply - WP11502.
References: alog 73706, 73367
Attached is the comparison of Oct/18/2023 22:30 UTC (current traces, no comb, thermistors cut off from Beckhoff but the heater is still driven by Beckhoff) and Oct/17/2023 22:33:40 (references, with 1.66Hz comb plus 1.235Hz SB to the 1.66Hz comb, thermistors and the heater connected to Beckhoff).
Red circles show 1.66Hz harmonics while blue ones show 1.235Hz harmonics or 1.235Hz SB to the 1.66Hz comb.
1.66Hz comb is visible in OMC QPDs (middle row, though only A is shown here, B is similar) at around 270-280 Hz as well as 177-180Hz, but only in PIT and YAW and NOT in NSUM (which I normalized by the DC value). This strongly suggests that this is either alignment or mode shape change.
We don't see anything at the fundamental 1.66Hz though, maybe the SNR is not enough. OTOH 1.235Hz appears in the current traces as well as references in the middle column. I haven't observed anything like 1.235Hz on the floor in Beckhoff, so I assume that this is NOT from the Beckhoff heater driver DAC. It's unclear if 1.235Hz is length-like (i.e. AM) signal coming from the IFO or if it's also alignment/modal shape change as the signal appears in PIT, YAW and NSUM.
In the second attachment, ASC-AS_A and AS_B DC don't see much except AS_A_DC_YAW sees something at 270.76Hz (top blue). The light power on these should be similar to OMC QPDs but WFS DC doesn't have the ISC whitening filter. ASC-AS_C doesn't see anything (2nd attachment bottom). This is what as AS_C is upstream of OM2 and the imprinting of the noise via the DARM loop shouldn't be strong at 270-280Hz, but note that it only receives 10 times smaller power than OMC QPDs, so the SNR is not as good anyway.
To measure the coupling TF from thermistor current to DARM, I'll route OMCD-DCPD_TEST_EXC to one of the thermistors (hot one at first, as it's mounted on the thermal compression ring which is closer to the electrical loop formed by the heater circuit).
(Sheila mentioned that when OM2 heater was off while the full connection to Beckhoff was still maintained, we didn't see the comb even though 1.66Hz thermistor switching should have been going on. It could be a magnetic coupling (e.g. the single loop electric circuit formed around the compression plate by the heater and its wires generates magnetic field that will push floppy thermistor wires that carry 1.66Hz current?) or maybe dependent on the tightness of the compression ring itself. We can easily distinguish the two by injecting into thermistors and turn off the heating . If the coupling change is instantaneous, it's probably the magnetic field.)
During the maintenance, I disconnected the H1:OMC-TEST_DCPD_EXC Lemo cable from the dual DB9 breakout panel in ISC R5 by HAM6 (so that it won't injected into the DCPD whitening), and used the TNC connector above LEMO to reroute the signal to the Thermistor 2 input at the back of the OM2 heater driver.
I used the breakout board at the back of the OM2 heater driver chassis as follows:
pin 9&11 (Thermistor 2 pin A and B) -> positive out (center conductor of TNC) of H1:OMC-TEST_DCPD_EXC.
pin 22&24 (Thermistor 2 pin C and D) -> negative out (isolated shell of TNC) of H1:OMC-TEST_DCPD_EXC.
pin 10&12 (Thermistor 1 pin A and B) -> bundled together on the breakout board but not connected to anywhere else.
pin 23&25 (Thermistor 1 pin C and D) -> bundled together on the breakout board but not connected to anywhere else.
pin 6&19 (Heater Driver voltage input) -> Beckhoff driver voltage.
(Before connecting it to the thermistors, I roughly measured the voltage across the H1:OMC-TEST_DCPD_EXC pins using a scope on a battery and it had ~15mV DC and maybe 5 to 10mV RMS.)
Max power rating of the thermistor is 18mW at 25C, it's substantially hotter than 25C for Thermistor 2, so I'll make sure NOT to exceed 5mW or so out of caution. Thermistor 2 resistance is more than 4kOhm but less than 5k when hot, so the maximum voltage would be something like sqrt(5mW*4kOhm)~4.5VRMS or ~12Vpp.
I didn't have time to do any measurement today so I'll use commissioning window.
Per WP11498 I replaced sw-gc-cdsadmin an old switch which resides in the gc comms room. This ties in some fmcs and cameras and passes them into CDS.
Tue Oct 31 10:11:52 2023 INFO: Fill completed in 11min 48secs
FYI, I've written up how the current overfill system works in DCC-T2300401
The RefCav transmission is quite low, so Jason had put in a work permit to do a remote alignment tweak. I'm reminding myself from alog 72222 what the procedure is (i.e. ISS stays on for this).
Attached is a several day trend of the RefCav transmission along with a temperature sensor inside the PSL enclosure. It looks like the temperature changed, but only by 0.3 F, and that perhaps started the drift down in transmission.
Notes to future self:
TITLE: 10/31 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Earthquake
OUTGOING OPERATOR: Tony
CURRENT ENVIRONMENT:
SEI_ENV state: EARTHQUAKE
Wind: 5mph Gusts, 3mph 5min avg
Primary useism: 0.35 μm/s
Secondary useism: 0.33 μm/s
QUICK SUMMARY:
- DM/CDS ok
- H1 is currently in EQ mode, following a 6.6 EQ from Chile
- IFO is sitting in IDLE ready to start the day of maintenance
With improvements in the past couple of months in the low frequency noise on both interferometers, and both now capable of achieving 160+ Mpc BNS range, I wanted to make a current comparison of the low frequency noise between H1 and L1 -- see the attached plot. For each interferometer, I selected a recent time where its BNS range was at/near its maximum, which is between 160-165 Mpc. The strain noise levels are nearly identical above 45 Hz. Below that, L1 has lower noise, by factors of 2-3 in the 10-30 Hz band.
Workstations were updated and rebooted. OS packages were updated. Conda packages were not changed.
TITLE: 10/31 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 157Mpc
INCOMING OPERATOR: Tony
SHIFT SUMMARY:
We've been locked the whole shift, 9:37 as of 07:00UTC. ITMY mode5 settings of FM6+FM10, G=+0.01 are working to damp the mode, it doesn't really affect mode6. Microseism is still elevated.
Full shift report can be found here: https://wiki.ligo.org/DetChar/DataQuality/DQShiftLHO20231023
Highlights:
STATE of H1: Observing at 157Mpc
The DIFF beatnote looks to be pretty stable over the past 5 months, the COMM beatnote has fluctuated more, and looks to be decaying over the past 10-20 days.
Attached is the COMM beatnote decaying with PR3, showing PR3 has moved ~1urad in PIT and YAW over the last week. Temperature has been oscillating more for the last week too. We have seen PR3 previously change with temperature 70008. HAM2 ISI is not moving.
Looking at locking yesterday, COMM beatnote was~ -19 before Initial alignment (t-cursor) and -16 afterwards. Sheila says that we usually have issues once it gets close to -20. Beatnotes last aligned in May 69907.
A recent suggestion is that we see if the BS coil drivers are a limit to our sensitivity now. The way we'd check is to put the BS back to it's acquisition state, and see if we see noise in DARM. Any excess noise can then be projected down with the ratio of the coil driver filters.
I took an old version of a coil state switching script (from userapps/isc/guardian/bs_m2_switch_fast.py), and added to it turning off the output gain (upon recommendation of LLO who suspects that as contributing to their lockloss). We don't have time to try it out today, but the script (attached here, as well as checked in to userapps) should be ready to try next week. We can consider running it on Tuesday morning just to check that there are no issues with it, but then actually try it Wednesday afternoon.
To turn the BS to it's actuation state, line 12 setting the 'actuatorState' variable should be set to 2, then run the script. Then, to go back to the lownoise setting, change that variable back to 3 and re-run the script.
Note that the BS suspension guardian can take us from the acquisition state to the low noise state gracefully, but it doesn't look like it's got built-in a graceful way to go back to the high noise state.
We were not locked (due to an earthquake) at the beginning of maintenance, so we have not yet tried this with the IFO locked.
However, I ran it a few times during maintenance and it seems to work. I did need to update the script to convince it to work with python3 (the version I started with was quite old). I attach the updated script here, but it's also still in userapps/isc/h1/guardian/bs_m2_switch_fast.py
I see some small motion in the BS optical lever siganls when I do the switching, particularly from the low noise state to the high range state (3->1->2), but I'm still working on convincing ndscope to lookback at data from the last time we went through ISC_LOCK state 552 (lownoise_coil_drivers) to see what scale of oplev motion we normally survive in that state.
I might try running this once or twice as we acquire lock after maintenance today, but otherwise this is probably as ready to go as possible in preparation for trying it during a commissioning time.
I activated Jenne's DARM2 boost from LHO:73745 (in FM2) at 10/27/2023 13:25:54. The FM2 change has been saved in SDF. The file that includes this new DARM2 boost filter is at/opt/rtcds/lho/h1/chans/filter_archive/h1omc/H1OMC_1382307261.txt. I've copied it to/ligo/svncommon/CalSVN/aligocalibration/trunk/Common/H1CalFilterArchive/h1omcfor the Cal group. The file has been committed to the SVN. The corresponding repo has been updated on the LHO cluster under the cal account too.
Added this FM2 boost to ISC_LOCK state LOWNOISE_LENGTH_CONTROL, FM2 is now turned on at the same time as DARM2 FM8 is.
This Calibration-affecting change has been recorded in the H1 Record of Real-Time Calibration Pipeline Parameter Changes (DCC T2300297).
The MY turbo station scroll pump was swapped with a newer pump with only 73.2 hours. The pump with 11395.3 hours will be rebuilt for future deployment.