TITLE: 01/03 Day Shift: 16:00-00:00 UTC (08:00-16:00 PST), all times posted in UTC
Summary: Calibration measurements utilizing Pcal to DARM transfer functions can be impacted by non-unity gain when making corrections for the modeled frequency response of the AA filtering. At LHO, the ER10/O2 model had an analog AA transfer function with non-unity gain based on the LTI measurements from ER8 (at LLO, the LTI model for ER10/O2 was normalized to 1). The analog AA model at LHO has a gain of ~0.99. Below we detail the impact on sensing function and actuation coefficients. In summary, the sensing function gain is ~1% larger than originally modeled, and the actuation coefficients are ~1% smaller. This would imply that the inspiral range is ~1% higher than currently predicted. This new understanding means that the analysis code needs to be re-run for the optical response parameters and actuation coefficients. The front-end calibration will need to be updated, and, finally, the GDS pipeline needs new filters generated and installed. Details: The calibration of the Pcal channels (ex: H1:CAL-PCALX_RX_PD_DQ) determines the watts reflecting from the ETM per count of the channel at DC. Whatever gain of the analog AA, this is already accounted for in this calibration procedure. This gain is implicitly accounted for when the value of the calibration is installed in the front-end filter module. A Pcal to DARM transfer function was previously understood as follows (note that PD calib., susnorm, m/N coeff are taken care of in the front end and (1+G)*(1/f^2) are taken care of in analyzing the measurements): DARM (IFO opt. resp.) (OMC DCPD TF) (AA(a) freq. resp.) (AA(a) gain) (AA(d) TF) ---- = ---------------------------------------------------------------------------------------------- PCAL (PD calib.) (AA(a) freq. resp.) (AA(a) gain) (AA(d) TF) (susnorm) (m/N coeff.) (1 + G) (1/f^2) where AA(a) is the analog AA, AA(d) is the digital AA, and "freq. resp." means the normalized transfer function, and G is the open loop gain. In the above (incorrect) understanding, the AA(a) and AA(d) terms cancel. The reason this is incorrect is that the AA(a) gain has an inverse in the PD calibration factor. So the real (correct) Pcal to DARM transfer function is: DARM (IFO opt. resp.) (OMC DCPD TF) (AA(a) freq. resp.) (AA(a) gain) (AA(d) TF) ---- = --------------------------------------------------------------------------------- . PCAL (PD calib.) (AA(a) freq. resp.) (AA(d) TF) (susnorm) (m/N coeff.) (1 + G) (1/f^2) Thus, to isolate the IFO optical response, we need to divide out the modeled AA(a) gain. Since the gain is ~0.99, then the gain of the optical response should go up by ~1%. The above equations are laid out in a graphical subway map schematic in G1501518-v14. The actuation coefficients will also be impacted by this, although the coefficients will be multiplied by the AA(a) gain so that the overall DARM OLG remains unchanged. I have pushed the changes to the DARM model code and scripts that account for this. Specifically: computeSensing.m (r4025) create_partial_td_filters.m (r4026) create_full_td_filters.m (r4027) fitDataToC_20161116.m (r4028). Re-running analysis of optical response parameters requires re-running the fitDataToC_20161116.m script first (with printing data to file), then running the fitCTF_mcmc.m script. Re-running analysis of actuator coefficients requires re-running actuatorCoefficients_Npct.m (with printing data to file), then re-running fitActCoefs_Npct.m. Hopefully this takes care of everything. Unfortunately, I cannot verify these changes with Matlab because I have no way of running Matlab offsite (need a network license). :(
1530 - hrs. local -> Kyle on site for daily inspection Found main entrance to OSB not locked -> I locked it. Noticed snow completely covering/blocking solar panels which charge the BT ion pump gauge batteries -> Took no action but lamented not not opting for the 24/7 diesel generators ;). Manually overfilled CP3 and CP4 (I have a problem - I know that now). Noticed that the red lamp which indicates the presence of 120VAC at the VPW outside water storage tank (DI water storage?) was not illuminated -> I investigated and confirmed that 120VAC was not present at the local heat trace thermistor? RTD? box(es). Also noticed that the GFCI receptacle which is on the same circuit and is used to supply trace heat to the line from the tank to the building penetration was tripped -> I reset it but the red lamp did not come on -> I measured 120VAC the GFCI receptacle after resetting it but still no red lamp. I opted not to start the WP process tonight but texted Bubba. Noticed vacuum gauge alarms still waiting for acknowledgment on the Operator's alarm computer but these are stale and left over from the PT-343 and PT-310 issues of a few days ago. Just took a call in the CR from Terry G. asking me to send out an LHO-all email that Hanford has announced a 2 hour delay to tomorrow's start of work day -> I'll send out an email now. 1915 hours local -> Kyle leaving site now
Starting CP3 fill. LLCV enabled. LLCV set to manual control. LLCV set to 50% open. Fill completed in 26 seconds. LLCV set back to 16.0% open. Starting CP4 fill. LLCV enabled. LLCV set to manual control. LLCV set to 70% open. Fill completed in 82 seconds. LLCV set back to 35.0% open.
I did not get the text message at the start of today's fill
Doh! Today I learned what that message on my cell phone meant - "Airplane Mode"! Wow! I had lots of messages, including the this one!
As found, the shop light used as a local heat source for CP5's LLCV ("control valve" for John) was on, the valve stem pointer was 60% < pointer < 70% and the pump level was ~75% full as indicated by CDS and 30" WC as per the magnahelic diff pressure gauge (mechanical level) -> I opened the LLCV bypass valve 1/2 turn to manually fill CP5 while I troubleshot (reached > 100% full in 35 minutes). I decided to fix the heat tape issue on this visit so we wouldn't need the light anymore. The trace heat switch in the VEA was on but no 120VAC at the heat tape -> found tripped breaker (CKT #1 in panel VEAC-01A???). I then confirmed 120VAC at the JCT box feeding the trace heat tapes - fixed (the breaker was likely tripped when the heat trace was installed a few years ago?) Next, I removed the electric actuator cover and found a 1/2" of standing water in the bottom of the housing (not frozen). One of the 24VDC wires had a crimp connector which was fully submerged. Measured 3.9VDC at the terminal strip inside of the housing. I noticed that the 1/2" plastic liquid-tight conduit elbow had been cross-threaded into the actuator housing such that it wasn't fully threaded and, as such, wasn't compressing the O-ring, i.e., was not making a water-tight seal -> I lifted the wires and removed and re-threaded this fitting (I should have tried to find a new one). I also followed Gerardo's lead and used electrical tape to seal all of the liquid-tight joints - Brrrrr.. cold fingers makes working with 18awg wires in tight spaces and electrical tape a humiliating task!
Back in the "oh, so warm" VEA I found the blown fuse for XV300 terminal 212 and replaced it -> Confirmed 24VDC in actuator housing - fixed! I stroked the LLCV to 0% open via MANUAL mode in CDS (note, I had to come back to the Fuching Corner Station to do this because I couldn't navigate the Beckhoff screens! Argh! I feel like a stranger in my own home. No more changes/upgrades people!) and confirmed (back at the X-mid) that the valve stem pointer was at 0% etc..
Leaving in PID control
The pending leap second LED cleared at 16:00 PST indicating the leap second was applied correctly. Disappointingly I did not see the seconds indicator do the 58, 59, 60, 00, 01 sequence, it looks like it stayed on 00 for two seconds.
It did what it was supposed to do, programmed that way to stay on for 2 seconds.
The computer h1susauxh34 died at 14:57 PST Saturday. It was powered up, but had a blank console and not reachable via network. Since this is not on the Dolphin fabric, I power cycled it at 15:53 PST and it came back correctly.
I'll stop in and fix after lunch, 90 minutes or so.
Started receiving text messages about PT343 CC @ 9:13 PM (PST), it turns out that somehow ithe CC turned off, I logged in to turn it back on (I did a handfull of toggles (on->off->on->off....->on)) but CC did not comeback on scale, I will leave it ON.
Uh, is PT310B being off the result of "clicking" the wrong button or is this Russian hacking!
On site to show family around, and PT-343 and PT-310 are alarming on the Ops alarm computer, and reading digital zeros on the vac overview screen. It looks like Team Vacuum is aware and monitoring, so I won't call anyone.
PSL dust is alarming too, but that seems like its usual M.O. lately.
As found, neither CP3 nor CP4 showing vapor at exhaust. CP3 had frosted exhaust line (typical for environment, shaded). CP4's exhaust line not frosted (typical for environment, in sunshine). As previously mentioned by Dave B., the T/C wires fed into CP4's exhaust have an excessive loop which will get snagged by tumbleweeds eventually -> Manually overfilled CP4 by opening its LLCV bypass valve 1/2 turn -> LN2 at exhaust outlet in 15 seconds. -> Manually overfilled CP3, by opening its LLCV bypass valve 1/2 turn -> LN2 at exhaust outlet in 15 seconds as well. Leaving CP3 LLCV %open for "others" to fiddle with remotely.
Bottle pressure of UHP N2 bottle used to slightly pressurize CP4's clogged sensing line (20 psi) now at 1000 psi.
~1340 - 1345 hours local -> Kyle in and out of LVEA
Rotating shaft vacuum pumps ("my girls") used for PT180 experiment are cool and quiet ("and happy - a man knows these things!") No sign of gauge drift at this point.
1410 hrs. local -> Kyle leaving site now.
(This is for yesterday during the Holiday Site Inspection for 12/29.)
Topped off the Crystal Chiller with 240mL of water. (FYI: It was lasted filled on 12/25 by Jason with 175mL)
No alarm for the Diode Chiller.
This closes FAMIS #6503.
Starting CP3 fill. TC A error. TC B error. Fill aborted. Starting CP4 fill. LLCV enabled. LLCV set to manual control. LLCV set to 70% open. Fill completed in 90 seconds. TC A did not register fill. LLCV set back to 35.0% open.
I logged in remotely and changed the setting for CP3 to 16% from 18%.
CP3 auto fill did not start, here are the logs
Starting CP3 fill. TC A error. TC B error. Fill aborted.
Starting CP4 fill. LLCV enabled. LLCV set to manual control. LLCV set to 70% open. Fill completed in 90 seconds. TC A did not register fill. LLCV set back to 35.0% open.
The code looks at the TC-A and TC-B temps before starting, for CP3 if they are below -20C the script will abort.
Looking at CP3 StripTool (attached) it looks like the script ran when the temps were in their low spike.
VAC team are looking into the issue.
CP4 filled normally.
Apollo was on site Tuesday and Wednesday of this week to begin the HVAC Controls Upgrade Project,specifically at the LSB. They have ~ 90% of the hardware installed in that building and plan on trying to finish up there possibly next week. They have started balancing the system and are writing the programs for the new system. Huge difference already.
Summary: shutting down the main HVAC system increased the range by 2-3 Mpc. The shutdown produced a large change in DARM, 8-10 Hz, and ~ 3% change between 48-130 Hz. The feature in the 8-10 Hz band may be due to turbulence in the Y-end chilled water flow and would probably be reduced by lowering the frequency of the VFD at Y-end. It is not yet known if this will reduce the noise in the 48-130 Hz band.
Thursday, Dec. 22, before we shut down, I cycled the main HVAC multiple times in order to see if it was costing us range. In the blue “off” periods in the figures, the chiller pad chillers and water pumps were off, the turbines were off at EX, EY, and CS, and the OSB fan was off. Figure 1 shows that the range improved by 2 to 3 Mpc during the “off” periods (we were averaging about 70 Mpc).
Figure 2 shows that the DARM spectrum improved in the 6-18Hz region, by a large factor between 8 and 10 Hz, and by roughly 3% in the 48-130 Hz regions. It is not clear whether the noise produced by the HVAC in the 48-130 Hz band is produced by vibration at lower frequency or by direct coupling. Low coherence with vibration sensors (< 1% in most of the 48-130 Hz band), and the observation that there is little change in peaks in that band that are known to be driven by vibration, suggest that the increased noise in this band is produced by vibration at lower frequencies and not by linear coupling. However, a preliminary look at PEM injection suggest that it is not impossible that the noise in the 48-130 Hz band is produced by direct vibration coupling. We will investigate this with further analysis of data from PEM injections.
The biggest difference between “on” and “off” was in the 8 -10 Hz band (Figure 2). Figure 3 shows that Y-motion of ETMY ST2 is quite coherent with DARM, while X-motion at ETMX is not coherent with DARM. The CS is also not as coherent at these frequencies. The chilled water flow at EY has previously been shown to produce vibration in this band from turbulence (https://alog.ligo-wa.caltech.edu/aLOG/index.php?callRep=11466), so reducing the setting of the variable frequency drive below the current 50Hz level (I recommend 35-45Hz in the linked log) might reduce the 8-10 Hz region of DARM. Further analysis of PEM injections and additional injections at 9 Hz at EY would help us determine if the 8-10 Hz drive at Y-end is also responsible for the noise in the 48-130 Hz band.
Shutdown times:
UTC Dec. 22
21:24:00 shutdown starts, 21:26:00 complete
21:34:00 startup starts, 21:36:40 complete
21:44:00 shutdown starts, 21:45:40 complete
21:54:00 startup starts, 21:55:40 complete
22:04:00 shutdown starts, 22:05:40 complete
22:14:00 startup starts, 22:15:40 complete
Given the large coherence between EY GS13 and DARM in the 8-10Hz band, can this be used to subtract out the noise in the affected band? perhaps in the F?E or in the calibration pipeline? Or would it be easier to eliminate the noise at its source?
I was wondering if the coherence between DARM and the GS13s could be somehow caused by our feedback of DARM to the ETMY suspension, but I think that it is not and that the coherence is probably due to noise from EY coupling to DARM as Robert suggests.
I found a 4 hour period starting at 11:51 UTC on Dec 1st when we had the interferometer locked on ETMX. The coherence between DARM and the GS13s is pretty similar when we are locked on ETMX. I wasn't able to plot data from NDS2 and NDS1 on the same dtt template, but the noise was higher in the lock from Dec 1st, which explains the somewhat lower coherence. In the attached screenshot both plots are made with 30 averages, despite the DTT display that says the one taken today only has 2 averages.
The point is that the coherence is higher with ETMY GS13s than ETMX no matter which suspension we are feeding back to.
For reference to the full paths, these files live at: {CALSVN}/trunk/Runs/O2/DARMmodel/src/computeSensing.m {CALSVN}/trunk/Runs/O2/TDfilter/create_partial_td_filters.m {CALSVN}/trunk/Runs/O2/TDfilter/create_full_td_filters.m {CALSVN}/trunk/Runs/ER10/H1/Scripts/PCAL/fitDataToC_20161116.m {CALSVN}/trunk/Runs/ER10/H1/Scripts/PCAL/fitCTF_mcmc.m {CALSVN}/trunk/Runs/ER10/H1/Scripts/FullIFOActuatorTFs/actuatorCoefficients_Npct.m {CALSVN}/trunk/Runs/ER10/H1/Scripts/FullIFOActuatorTFs/fitActCoefs_Npct.m