Today the CDS overview screen looks like it has a bad case of measles (see attached). This is due to the h1ioplsc0 model ADC/TIM glitching now it has a faster computer. Like the end station SUS, some times the glitch propagates into the IPC channels and in turn glitches the receiver models. Because LSC touches more models than end station SUS, these glitches are more prominent.
Last night I added the LSC models to my script which issues diag reset to fast computer models and receivers of end-station SUS every minute. I'll extend it to cover receivers of LSC.
Jeff B, Dave:
I have re-written the check_dust_monitors_are_working script to make the output more readable.
The script defaults to checking the past 12 hours. Additionally it can take a second argument to check for N hours ending M hours ago, where N and M are the two arguments given. For each dust monitor, it internally checks both 0.5um and 0.3um dust counts. If both have shown no variation over the time period provided (i.e. both have zero standard deviation) then a warning message is printed, otherwise an ok message is printed.
Here is a "all is good" example for the past 12 hours:
check_dust_monitors_are_working
Checking dust monitors for possible problems over a period of 12 hours ending 0 hours ago...
H1:PEM-CS_DUST_LVEA2 OK
H1:PEM-CS_DUST_LVEA3 OK
H1:PEM-CS_DUST_LVEA4 OK
H1:PEM-CS_DUST_LVEA5 OK
H1:PEM-CS_DUST_LVEA6 OK
H1:PEM-CS_DUST_LVEA10 OK
H1:PEM-CS_DUST_LVEA30 OK
H1:PEM-CS_DUST_PSL101 OK
H1:PEM-CS_DUST_PSL102 OK
H1:PEM-EX_DUST_VEA1 OK
H1:PEM-EY_DUST_VEA1 OK
done
Here is an example with HAM6's monitor in error, prior to this morning's restart of the LEAV6‌ dust monitor:
check_dust_monitors_are_working 12 12
Checking dust monitors for possible problems over a period of 12 hours ending 12 hours ago...
H1:PEM-CS_DUST_LVEA2 OK
H1:PEM-CS_DUST_LVEA3 OK
H1:PEM-CS_DUST_LVEA4 OK
H1:PEM-CS_DUST_LVEA5 OK
H1:PEM-CS_DUST_LVEA6 WARNING: dust counts did not change, please investigate
H1:PEM-CS_DUST_LVEA10 OK
H1:PEM-CS_DUST_LVEA30 OK
H1:PEM-CS_DUST_PSL101 OK
H1:PEM-CS_DUST_PSL102 OK
H1:PEM-EX_DUST_VEA1 OK
H1:PEM-EY_DUST_VEA1 OK
done
Travis, Ed, Greg
We went down the x-arm to check if the cryopump baffle ring was grounded like the issue that Livingston had, alog 33667, where the ring was touching the copper pieces. One side was not centered very well, but was still clear of the copper block, the other side was pretty well centered. A quick bump test showed the baffle ring floating freely.
Images 667 & 669 (W side) show that there is no coupling between the copper of the baffle and the steel of the beam tube. It's pretty tight, but there is a gap. Images 670 & 671 are of the E side and there is a significant gap there. Your welcome for the "bump test". :P
Power cycled at 17:44. It's working properly again at this time.
Both doors on HAM 4 were removed this morning. HEPIs are covered also.
Everythin seems to be in working order this morning.
It seems that the data from the script output differed from the MEDM screen. Regarding HAM6: The terminal data showed 0.00 in both ranges despite the MEDM screen showing normal operation. CDS will look into this.
TITLE: 10/13 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Planned Engineering
OUTGOING OPERATOR: None
CURRENT ENVIRONMENT:
Wind: 5mph Gusts, 3mph 5min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.19 μm/s
QUICK SUMMARY:
Kyle R., Rakesh K. Today we decoupled the 10" gate valve that isolates the YBM Main Turbo Pump (YBM MTP) from the Y-beam manifold. Next, we removed the 10" valve from the YBM pump port. For tonight, we are leaving the YBM 10" pump port covered in UHV AL Foil. Tomorrow we will install the valve and should make some progress towards re-coupling the the turbo to the valve.
With help from Mark and Tyler, Betsy and I reinstalled the ITMx lower structure into BSC3 this morning. In the afternoon, we worked on resuspending it to begin assessing alignment. Tomorrow, we will check the alignment at the OpLev viewport and begin adjustments.
TITLE: 10/12 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Planned Engineering
INCOMING OPERATOR: None
SHIFT SUMMARY:
LOG:
15:15 Vent meeting
15:57 Travis and Betsy ou to the biergarten
16:00 Jason and TJ out to LVEA
17:44 Jason out to HAM5
18:05 Travis and Betsy out
18:29 Jason and Garilynn out
18:40 TJ out for lunch
18:54 Bubba and crew taking the HAM4 South door off (started earlier, not sure when)
19:00 Dave B having Dolphin issues and the CDS overview is quite red with TCS Chiller imminent on Verbal Alarms and the PSL PMC not unlocked
19:05 Dave B out to EY to investigate Dolphin issue.
19:23 Dave back
20:00 Untripping IMs
20:02 Untripping MCs
20:03 Untripping PRs
20:05 Untripping BS
20:06 Untripping SRs20:12 Reset PSL EPICS alarm and readjusted ISS Diffracted Power
20:25 Marc to MY
20:39 Cheryl out to HAM2
20:44 Jason out to BSC1
20:57 Marc back
22:10 Betsy and Travis back out to biergarten
This is a good time to summarize the CDS Computing changes completed for Squeezer install
h1asc0:
Removed 8th ADC.
Removed 2nd 16bit-DAC
Removed h1sushtts model
Removed h1susauxasc0 model
Modified h1asc and h1ascimc models to use 1st 16bit-DAC instead of 2nd card
Modified h1iopasc0 model to reflect new IO Chassis hardware configuration
Added h1sqzwfs model (dcuid=77)
h1lsc0:
Upgraded computer to fast 10 core machine
Added ADC card
modified h1ioplsc0 to reflect new IO Chassis hardware configuration
Added h1sqz model (dcuid=76)
h1sush56:
Added two 16bit-DAC cards
Modified h1susomc model to add OM1,2,3 (originally in h1sushtts)
Added new h1susopo model (dcuid=22) for VOPO, OFI, ZM1,2
Modified h1iopsush45 to reflect new IO Chassis hardware configuration
h1sush2b:
Added 16bit-DAC card
Added h1sushtts model (dcuid=21) for RM1,2
Modified h1iopsush2b to reflect new IO Chassis hardware configuration
h1susauxh56:
Added ADC card
Modified h1susauxh56 model, added SQZ aux channels
Modified h1iopsusauxh56 to reflect new IO Chassis hardware configuration
h1susauxh2:
Modified h1susauxh2 model to add RM1,2 aux channels
I have created a placeholder h1sqz model and started it on h1lsc0. It has dcu-id=76, a processing rate of 16384Hz and runs on the 7th core of this new 10 core machine.
I have not yet added h1sqz to the DAQ. I'll do that when we next restart the DAQ.
I have added the model to the CDS Overview MEDM (see attached).
I believe this completes the new parts needed in CDS computing for Squeezer install.
BTW: after reporting how to annotate PNGs using GIMP, Vern countered with a suggestion of using INKSCAPE instead. So in the spirit of impartiality here are the inkscape instructions
Pit Slider | Yaw Slider | P OSEM | Y OSEM | |
MC1 | 1112.7 | -2085.2 | -104 | -1136 |
MC2 | 455 | -594 | 516 | -704 |
MC3 | -3911 | -490 | -3606 | 500 |
IM1 | 0 | 0 | 14 | -182 |
IM2 | 0 | 2000 | 62 | -1016 |
IM3 | 18300 | 5000 | 1485 | 99 |
IM4 | 27000 | 0 | -2718 | -1073 |
PRM Aligned | -1115 | -202 | -1064 | -132 |
PRM Mis-Aligned | -1509 | 544 | -945 | -2023 |
The pitch offsets on IM3 and IM4 were necessary to align the beam to PR2 on 10/6, but not typical.
Additionally, IM3 bolts had not yet been torqued, but have been now, so it's alignment values may change by ~100's of counts.
This will be revisited with IO's next opportunity to to have the beam from the PSL.
Krishna, Jim
This is a slight variation on an earlier duty cycle analysis by Jim. I'm trying to establish how the new ISI-Stage1 control scheme implemented in O2 at LHO benefited the interferometer. As a reminder, in O1, we only used feedback from the Stage 1 seismometer and switched between the 45/90 mHz blends to combat microseism/wind respectively. In O2 we used 'tilt-subtracted' feedforward at low-frequencies and 250 mHz blends as the nominal configuration on all platforms including the HAMs. The data lives in: SeiSVN/seismic/Common/Data/LHO_O1_O2_duty_cycle_data
The first attachment shows plots for duty cycle versus wind for O1/O2. It uses the minute trends of ISC Lock State and the ETMY windspeed (max) signal. The first page simply shows the distribution of wind - fraction of time windspeeds were in a given bin (bins were ~2 mph) during O1 and O2. The second page shows the fraction of the time the interferometer was locked at a given windspeed. Not only is there a clear improvement in O2, but the curve looks flat up to a windspeed of ~30 mph unlike in O1. The overall duty cycle in O2 seems to have suffered a bit, possibly due to other reasons. Pages 3 and 4 show similar plots, but only comparing the 45 mHz blends used in O1, which are still the default configuration at LLO. Again, it is interesting to note the downward trend on page 4 for the 45 mHz blend, which suggests that even 10-20 mph winds would begin to impact the interferometer.
The second attachment has very similar plots for duty cycle versus microseism velocity, using the band-limited-rms ITMY_Z (max) signal in the microseism band. The O2 configuration looks better once again and there is a similar trend of nearly flat duty cycle up to ~1300 nm/s velocities in O2. The distribution of the velocities looks odd/different, partly because of the inclusion of Hanford summers in O2, which are very quiet in the microseism.
I'm attaching the cumulative distributions of the wind and microseism (z) velocities (max of minute trends), so for example on the wind plot, the y-axis means that the wind is above ~15 mph about 15% of the time.
WP7162 Arnab, Dave:
This morning we upgraded h1lsc0 to a faster model (same model as h1suse[x,y]). The reason for the upgrade is not that we need the faster cores per-se, rather we need the additional cores provided to run the new h1sqz LSC squeezer model. The original was a 6 core machine, allowing 4 user models. The new is a 10 core machine, allowing 8 user models.
To expedite the install, we took the DTS x1susex computer, added a second GeFanuc 5565 card and used that for h1lsc0.
Unfortunately despite our best efforts, the upgrade glitched the corner station Dolphin fabric, and the models had to be restarted on all the Dolphin'ed corner station front ends, which includes the PSL.
We have verified that X-Arm RFM IPC signals go to ETMX and Y-Arm to ETMY.
The h1omc model sends DARM_CTRL to ETMX with a matrix element of +1.0 and to ETMY with an element of -1.0. The DARM_CTRL signal is currently the CAL_LINE_SUM signal, a ~30Hz sine wave with a small amplitude of 0.4 counts. The attached dataviewer plot shows the sending trace (CH3: H1:LSC-CAL_LINE_SUM) the receiver on h1susetmx (CH4: H1:SUS-ETMX_L3_ISCINF_L_IN1) and the receiver on h1susetmy (Ch5: H1:SUS-ETMY_L3_ISCINF_L_IN1). The plot is 1/16th of a second on the time axis. As can be seen, ETMX is in phase with CAL_LINE_SUM and ETMY is 180deg out of phase.
The relevant part of h1omc model and the matrix settings are shown in an attachment.
To be absolutely sure of which Dolphin port the cable should be connected to, we drove out to EY and verified with h1susey.
Model processing time changes resulting from computer upgrade (see attached minute trend plot).
Ch1: h1ioplsc0. Did not noticably speed up, in fact looks like it may have slowed by 1uS. min-max range has tightened up.
Ch2: h1lsc: Sped up from 38 +/- 4uS to 25 +/- 0uS (12us (32%) faster with very little jitter)
Ch3: h1omc: Sped up from 22 +/-2uS to 15 +/-0uS (7uS (32%) faster with very little jitter)
Ch4: h1lscaux: Sped up from 19 +/-1uS to 15 +/-0uS (4uS (21%) faster with very little jitter)
Ch5: h1omcpi: Sped up from 8 +/-1uS to 6 +/0uS (2uS (25%) faster with no jitter)
S. Dwyer, J. Kissel While inspecting HAM6 regarding the broken OMC REFL Beam Diverter Dump, we took the opportunity to check out the fast shutter. We saw several things of concern: (1) The cabling that emanates from the shutter itself looks (and has been previously confirmed to be) very close to the main IFO beam path. Koji indicated in his LHO aLOG 28969 that "the clearance does not look great in the [above linked] photo but in reality the beam is clearly away from the wire." (2) One of these wires (the wire closer to the IFO beam) has a kink in it, but no visible burn marks, so we suspect this is from mechanical manipulation during install. (3) OM3's readout cables are kinked in a stressed position to make room for the fast shutter, which is a little forward (-Y, away from the beam splitter) of the position indicated in the drawing likely because of this interference. (4) Some small fleck of particulate on the inside of the "toaster" face, on the HR (back towards OM1) side of the shutter. I attach picture collections of these things (admittedly the particulate pictures are not great -- we tried lots of times to get a good picture but failed). Hopes: Re (1): Can we find a way to route these cables below the beam line, instead of surrounding it? Re (2): Same as (1) Re (3): Can we replace OM3's OSEM cables with a 90 deg backshell, so as to clear room for the fast shutter and relieve stress on the cable? Re (4): hopefully this is not from the HR surface of the fast shutter optic
We were motivated to look at the fast shutter wires by the incident on July 17th where the wire from the fast shutter must have been clipping the to OM1 from HAM5. My original alog about this might not have spelled things out well enough, so here are some plots.
The first plot is from July 17 2017 5:51:00 UTC, this was after Jim, Cheryl and I figured out that the fast shutter was malfunctioning. We locked DRMI, and opened and closed the fast shutter several times. The top panel shows the state of the shutter, 0 is open 1 is closed. The second panel shows AS_A_SUM, which is downstream of the shutter, and goes to zero when the shutter is blocking the beam as it should. The third panel shows AS_C, which is upstream of the shutter but downstream of the place where the wire is close to the beam (check Jeff's annotated photo). You can see that moving the shutter causes dips in the amount of light on AS_C, and that the wire must land in a slightly different place in the HAM5 to OM1 beam causing a different level of light to be seen on AS_C
The second attachment shows that the shutter did seem to block the beam going to the AS WFS in the July 16th lockloss before we had this malfunction. Chandra also checked vacuum pressures for a spike in HAM6 pressures similar to what happened when we burned the OMC in August 2016, and saw nothing. I had been wondering if the fast shutter might have failed to durring a lockloss where the OMC was unlocked, which could result in a lot of power being on the beam dump. It seems like this didn't happen on July 16th.
Note, we played with this shutter cable last in Aug 2016. We struggled with getting this wire away from the beam at that time. The pictures in the log from Aug 2016 28969 show that we left the wire in a larger arc than the pictures show now. I suppose it's not so surprising that the wire has maybe migrated into the beam path over the numerous cycles over the last year.