Robert has turned of the corner station air handlers for some PEM measurements, they should be off for about an hour.
Trending the h1iopsusauxh2 cpu meter (attached) shows that the system crashed at 08:53 UTC (01:53 PDT) and was restored 09:40 UTC (02:40 PDT). I've rounded this up to a 1 hour downtime for the FRS accounting.
While getting these stats I've noticed a problem with h1edc's trending of its own channel-connection counters. I've opened FRS12776.
TITLE: 04/20 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 108Mpc
INCOMING OPERATOR: Jim
SHIFT SUMMARY:Other than h1susauxh2 going down, it was an uneventful shift. Locked for almost 14hrs.
LOG:
The interferometer stayed locked, but we cannot go to observing because of channel connection errors.
Cannot ping h1susauxh2 computer, TJ is rebooting using the front panel RESET button.
h1edc has 17 disconnected channels from the 2 models running on h1susauxh2:
Reboot was successful, models are running. All eight ADC cards are seen by IOP.
I called Dave and the problem was remedied with a simple reset from the front panel of the h1susauxh2 machine. This was possible since this machine is not in the Dolphin network.
We are back to Observing
h1edc green again. I've cleared the DAQ-CRC counters on h1[iop]susauxh2 models.
Opened FRS12775 for this.
TITLE: 04/20 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 107Mpc
OUTGOING OPERATOR: Travis
CURRENT ENVIRONMENT:
Wind: 8mph Gusts, 6mph 5min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.17 μm/s
QUICK SUMMARY: Favorable environment on a 6hr lock with triple coincidence.
TITLE: 04/20 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 109Mpc
INCOMING OPERATOR: TJ
SHIFT SUMMARY: No issues after relocking early in the shift. Have been locked in Observing for ~6 hours currently.
LOG:
23:48 Chandra back
0:00 PEM crew done for the day
Sharan Banagiri, Philippe Nguyen, Robert Schofield
We found low-level coupling when we shook the reduction flange by the ITMX optical levers, we repeated several impulse injections to give higher SNR, and we tried to narrow down, without success, the coupling site that produces a 48 Hz peak in DARM when we make broad band acoustic injections.
Times: 21:20 - 23:25 UTC
Ran through initial alignment after the lockloss following a ~30 hour lock stretch. After IA, H1 locked first try with no issues. Accepted one SDF diff for MC2 (see screenshot).
Cause likely known.
TITLE: 04/19 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Commissioning
OUTGOING OPERATOR: Jeff
CURRENT ENVIRONMENT:
Wind: 23mph Gusts, 17mph 5min avg
Primary useism: 0.14 μm/s
Secondary useism: 0.19 μm/s
QUICK SUMMARY: Currently in Commissioning for PEM injections.
23:10 HAM6 ISI WD tripped (assuming this is PEM), reset WD and did not lose lock
23:11 HAM6 ISI WD tripped, reset and maintained lock
J. Kissel, L. Sun While Lilli was confirming that the calibration group's estimate of the calibration uncertainty against our PCAL to DELTAL_EXTERNAL transfer function sweeps (e.g. LHO aLOG 48535) which requires compensation for the artificial delay we add between the actuation and sensing paths, H1:CAL-CS_DARM_CTRL_DELAY_CYCLES (for explanation as to why, check out T1900169), she found a great deal of confusion when processing sweeps when this delay was 7.0 (prior to Apr 16th) vs. when it was 8.0 (after Apr 16th). Namely, she found that compensating for 8.0 created a HUGE reported systematic error, where as 7.0 did not. Even worse, this did not reconcile with our empirical evidence from ER14, when we would change this delay from 7.0 to 8.0 to 9.0, and it showed *no* impact on the measured transfer function. Today, we measured the transfer function from just before this integer delay part to just after the delay part set at several integer values in order to settle this confusion once and for all. We were surprised to find that the transfer function behaved as expected only up to 7.0 clock cycles. Above which (we checked at 8.0, 9.0, 10.0) the transfer function remained as though there was only a 7.0 clock cycle delay. The first attachment shows the measurement. The second attachment shows the MEDM screen location of this parameter. The thirt shows the simulink location in the DARM block of the CAL_CS_MASTER.mdl library. Thank the flying spaghetti monster that the right answer was ~7.0 clock cycles! There is obviously some bug in this part, or limitations that we don't understand. The code lives here: /opt/rtcds/userapps/release/cds/common/src/RING_BUFFER.c We leave the delay at 8.0 clock cycles and email Joe Betzwieser, the author of the code. Note -- LLO is unaffected because they've been using a thiran filter since before O2 so as to get a non-integer clock cycle delay -- i.e. the CTRL DELAY filter. We haven't switched over to using that filter because when we recently created a thiran filter (LHO aLOG 48008), and tried to install it (LHO aLOG 48012) it "didn't work" in that it "created more systematic error than the change between 7.0 and 8.0 clock cycles," but that statement was made before we understand how to correct the PCAL to DELTAL EXTERNAL transfer function. It may be worth exploring again, it may not be. we'll see what the fallout of the bug fix is.
Good first half of the shift. IFO has been locked for 25 hours, with a range between 106 and 111Mpc.
I dropped it out of Observing for 5 minutes (16:20 (08:20) to 16:25 (08:25)) to damp ETM-X violin mode 13. Damping was effective, and no additional violin modes have rung up.
Weather is showing the effects of a cold front moving through to the northeast of the area. As the side bands pass through the wind has piped up to 8 to 12 mph, with gusts getting close to 20mph. Microseism is following the wind.
PEM injections are scheduled to start at 21:00 (14:00).
No other issues or problems to report.
Gerardo M., Kyle R.
Following the recent LGV11 actuation failure, analysis and recovery (here is a good place to recap -> https://alog.ligo-la.caltech.edu/aLOG/index.php?callRep=42304), the Project Vacuum Team concluded that all legacy 44" electrically actuated gate valves (at both sites) would need to be visually inspected. This process was started and is continuing as opportunities present themselves (maintenance days). Today we inspected WGV15, WGV14, WGV9, WGV10, WGV17 and WGV18.
Today's results are potentially troubling -> Both SET SCREWS in the CLUTCH BODY of WGV10 were found to be very loose -> I tightened these (2 turns) but was torque-limited to a few inch*pounds by the L-key used. I don't have a torque specification even if I did have the proper tool. Also, the clearance between the RETAINING NUT and LIMIT SWITCH PULLEY was noticeably greater than that of the other valves. By chance, one of the two DRIVE SHAFT BALL NUT SET SCREWS was accessible on WGV14 and it was found to be quite loose -> This was tightened using an L-key to a few inch pounds. The CLUTCH BODY of WGV17 differed from any of the other units seen thus far in that its SET SCREWS are oriented (machined) at 90 degrees to each other versus 180 degrees for the other units. Finally, WGV18's CLUTCH PRE-LOAD was at "8" on the graduated scale. This compares to the more typical values of 4 or 5 -> This may be due to the placement of the graduated scale on this unit and may not translate into a meaningful PRE-LOAD difference as the number of exposed thread crests seen is typical of the other units.
The archived results of these inspections is kept in the DCC in Q1800019 (thanks Stephen A.).
Today I revisited GV10 (this time with hand held lighting) and found that the "gap" initially observed was nothing more than the chamfer machined on the retaining nut.
In alog 48378, when comparing the broadband + sweeps measurements (CALCS) to pyDARM (GDS), an overall 40-microsecond delay is required to get make the two results consistent with each other, which is not understood.
After step-by-step investigation and fixing a few bugs in the following two scripts, the 40-microsecond delay mystery is solved. Now by applying the R_GDS/R_CALCS corrections, the blue dots are corrected to green dots (see attachments). Details are documented in T1900169
Broadband: /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Scripts/CALCS_FE/process_broadband_pcal_20190410.py
Sweep-sine: /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O3/H1/Scripts/FullIFOSensingTFs/process_sweep_pcal2darm_20190410.py
The steps are: (see details in the scripts)
1) The delta_C and delta_A are obtained by using the following variables:
delta_C = sensProd.calcsResid2gdsInvSensOut
delta_A = actProd.calcs2gdsResidActFiltOut_NoDAQDownSample_removeCALCSdelay #The DAQ downsampling is not used in CALCS, so we do not include it
2) The delay between C and A in CALCS is not seen by GDS. So we need to correct it.
R_cor = (1/delta_C) * (1+C*A*D) / (1 + C*A*D / (delta_C * delta_A / delay_between_C_and_A))
When taking the 0410 measurements at LHO, a 7-cycle delay was installed. (not 9 cycles updated after processing the measurements).
Also we have fixed the script of computing relative delay between A and C (Scripts/CALCS_FE/compute_relativedelay_AvsC_20190404.py) Now the correct delay estimate for 0404 model is 7.7 cycles. (See attachment) We will install 8 cycles.
3) Apply dewhitening (from CS_DARM_DELTAL_EXTERNAL_WHITEN filter)
4) Apply the total PCAL correction sensProd.pcalCorrection
The comparison between the sweep-sine measurements and the uncertainty envelope is also updated (see attachment).
I have added the option of plotting PCAL to DARM measurements on top of the uncertainties in RRNom.py. The measurement data file needs to be specified. Other temporary scripts are removed from svn.
See examples in aligocalibration/trunk/Common/pyDARM/README
I've bypassed this alarm for a couple of hours.
Bypass will expire:
Sat Apr 20 11:53:55 PDT 2019
For channel(s):
H0:FMC-EY_CY_H2O_PUMPSTAT