For reference, here is an email I sent today to lho-all reporting recent CDS backup systems problems, short term and long term solutions:
Over the past few weeks we have developed problems with the CDS backup systems, namely:
Yesterday, the masses on the STS2s were centered but I should have monitored the mass positions more in real time as the centering doesn't always take the first or second time around. While we did improve things maybe, there are still problems: six of the nine masses (3 masses 3 sts2s) were out of spec but after centering, there are still 3 of the 9 out of spec.
At EndX, the STS2 had two masses out of spec by about 50% (7500 cts vs 5000); after centering one of those came into spec but the other overshot and is now out by a factor of 5! Attached is ASDs of before and after. The refs are from 2am Tuesday and the currents are from 2am this morning. The useism is likely a bit higher early this morning (not as high as it is right now though.) There is not anything glaringly wrong to me here. In general the current traces are higher than the references but again that could just be the actual ground motion. I thought such a large mass position problem would be obvious but it is not to me. I'll do a comparison of LVEA instruments at the same time.
Here are two day trends of the three corner station STS2 Mass positions. The bottom row are the U mass, middle are Vs and the top are W masses. The first column, B2 is the STS2-B (itmy,) the middle column is the B1, STS2-A located near HAM2, and the third column, B3 is the STS2-C near HAM5.
Currently, STS2-B is the unit used for all the corner station sensor corrections. Yesterday's centering step is clearly visible and please note that the U mass shifted alot but is still equally out of position. B1, the HAM2 unit is not in service and was not centered yesterday. I do not understand why there is a glitch to zero at the time. We did connect a DVM to the Pos Spigots...maybe we toggled the UVW/XYZ switch and that is the glitch. The third column shows the HAM5 B3 unit which we did center although it looks like we really did not need to do that. Please note the various noise levels on these channels and especially how that changed on the B3 V signal, whats up with that?
Here is the spectra comparison at ETMY where the zeroing seems to have improved all the mass positions although one still is out of spec by maybe 50%. SImilar story to the EndX assessment above: nothing obvious given the potentially different ground motion. However, if you compare the ENDX and ENDY spectra, it sure would appear that the ENDY sees much more tilt than ENDX.
Here is the promised same time comparison after centering for the STS-B and C units. Again, the centering on the C unit was probably not needed (we were testing the process) and the centering on the B unit just sent it to the other side of zero with no real improvement in position.
Couple thing from this. See how the traces are almost exactly the same from .1 to 6 hz on the Y dof? That suggests these two instruments are doing well in that channel. Below about 60mHz, tilt starts to get in the way here; above 6 hz, wavelengths bring about differences. On the X and Z axes, the difference in this region between the spectra suggests a problem in those dofs. We suspect STS2-B, the ITMY unit. Now this could be because of the poor centering or maybe the difficulty in centering is a common symptom of the underlying instrument problem.
Plot shows ETMX and ETMY Stage 1 X loaction and Y location over the last 24 hours.
During the last 24 hours, useism has increased from 0.4um/s to 1um/s.
ETMY ISI (left, top and bottom) shows a steady increase in the amplitude of the oscillations in X Location and Y Location.
ETMX ISI (right, top and bottom) shows an increase in the amplitude of the oscillations in X location and Y Location, however, the increase in amplitude of the signals, and the signals themselves are anything but "smooth."
Activity Log: All Times in UTC (PT) 08:00 (00:00) Take over from TJ 12:00 (04:00) Dozens of ETM-Y saturations alarms concurrent with RF45 noise. 12:45 (04:45) Temperature alarm on CS Zone 4 duct. 12:58 (04:58) Temperature alarm on CS Zone 1A duct. 13:27 (05:27) The ETM-Y/RF45 saturations have stopped. 15:19 (07:19) Three ETM-Y/RF45 saturations in 8 seconds. 15:44 (07:44) Bubba – Into the Cleaning area to get a headset 15:46 (07:46) Bubba – out of the cleaning area 16:00 (08:00) Turn over to Cheryl End of Shift Summary: Title: 12/22/2015, Owl Shift 08:00 – 16:00 (00:00 – 08:00) All times in UTC (PT) Support: Incoming Operator: Cheryl Shift Detail Summary: Starting at 12:00 (04:00) dozens of ETM-Y saturations with simultaneous RF45 saturations. These continued until 13:27 (05:27). There were a few additional saturations after 13:27. Other than the RF45 saturations, it was a good night for data collection. Seismic and microseism continue to increase, as does the wind.
Starting at 12:00 (04:00) getting dozens of RF45 and concurrent ETM-Y saturations. By 12:55 (04:55) the rate of saturations had slowed but continues.
The last saturation was at 13:27 (05:27).
A shift without drama so far, with the IFO locked and Observing with good range. Environmental conditions however are deteriorating. Wind speeds are up into the mid-teens, with several gusts over 20mph. Seismic trend is upward, with a few EY peeks well over 0.1um/s. Microseism also has an upward slope, centered around 0.8um/s. There are a couple of large storm systems moving through the Bering Sea and the Gulf of Alaska over the next few days. The data buoys on the Washington coast are recording near shore wave heights up to 16 feet and winds at near gale force (up to 38mph).
Title: 12/23/2015, Owl Shift 08:00 – 16:00 (00:00 – 08:00) All times in UTC (PT) State of H1: 08:00 (00:00), The IFO is in Observing mode for the past 2.5 hours. Power and range are at normal levels. Wind is currently a gentle breeze (8-12mph) and building. Seismic and microseism are both rising as well. Monitoring CP-3 N2 levels. Outgoing Operator: TJ
TITLE: 12/22: 00:00-08:00UTC (16:00-00:00 PDT), all times posted in UTC"
STATE Of H1: Observing at 80Mpc for 2.5hrs
SUPPORT: Evan G, Darkhan
SHIFT SUMMARY: Had one mysterious lock loss, but didn't have too much trouble getting it back up.
INCOMING OPERATOR: Jeff B
ACTIVITY LOG:
Back to Observing.
Lost lock a few times when trying to switch from PRMI to DRMI (shot attached). After I would lock PRMI, adjust to maximize the signals, then try to transfer, it wouldn't work so it would then try to lock DRMI. After a few minutes it would seem like it would lock but then look as though it was trying to transfer again. Hard for me to describe it. Also lost lost once at Coil Drivers, but other than that it wasn't anything out of the ordinary.
Not sure why yet, everything was looking great.
SudarshanK, DarkhanT, EvanG
Pcal PD Calibrationat at ENDY was completed today. The results are within the variation seen in the past. A more detail report coming soon.
TITLE: 12/22 Eve Shift: 00:00-08:00UTC (16:00-00:00 PDT), all times posted in UTC"
STATE Of H1:Observing at 78Mpc for ~1 hours
OUTGOING OPERATOR: Cheryl
QUICK SUMMARY: Seems to have been a quick recovery form maintenance day! Wind is <15mph, useis 0.5um/s, PSL 21.9, CW inj running.
State of H1: Low Noise, A2L measurements finished, now in Observe
JeffK and I talked to LLO, and they are working on a measurement, and expect to be back in Observe soon.
Help with Mantenance and Relocking: Kiwamu, JimW, Ed, JeffK, Jenne
Relocking Details/Issues:
Currently: H1 in Observe at 23:24:36UTC, and as of this alog, range is 80Mpc.
Ops Day Summary: 16:00-23:59UTC, 08:00-16:00PT
State of H1: Observe for 44 minutes, range is 80Mpc
Current state of the site:
I just notinced that CP3 LT200 is again 0!
It seemed to be working again earlier today, but seems to have failed again, or something.
There's an earlier algo/alogs from VAC with more info.
The volume for the alarms were so very quiet this morning I couldn't hear them, and the volume for the verbal alarms was reasonable, and of course, when I tried to raise the alarm volume, the verbal alarms were too loud.
JimB discovered that the sound settings on the computer are seperated into "alarms" and "computer" volume sliders, and the alarm volume was set very low.
I discovered that to have the alarm volume about the same as the verbal alarm volume, the volume on the alarms needs to be at max. and the verbal alarms need to be at about 1/2 of the total possible volume.
Picture attached.
I increased the power on the BS and SR3 oplev lasers this morning by ~5%. I used the voltage read out of the "Current Monitor" port on the back of the laser (this port monitors the laser diode current, outputs a voltage). The old and new values are listed below, as well as the new SUM counts. Since I had to open the coolers the lasers are housed in they will need ~4-6 hours to return to thermal equilibium; at that point I can assess whether or not the glitching has improved.
Links to todays DetChar pages (look at the SUM spectrograms, both standard and normalized, at the top of the page) for these oplevs, showing data after the power increase and subsequent return to thermal equilibrium (data starts at 0:00 UTC 12/23/2015, or 16:00 PST 12/22/2015): BS, SR3. As can bee seen, SR3 quieted up significantly (go back a day or two and compare), one more small power adjustment should take care of it; will do that at the next opportunity (probably the next maintenance day, 1/5/2016). The BS on the other hand has not seen much improvement. I'm starting to think maybe I've overshot the stable range... Will perform another adjustment (probably also on the next maintence day), this time targeting a region between my last 2 adjustments. I will keep WP5667 open as this work is still ongoing.
On a brighter note, I have been testing the laser removed from the BS oplev in the LSB optics lab and can find nothing wrong with it. I'm currently adjusting the TEC temperature setpoint to bring the stable region to a more useful power level and the laser will be ready for re-installation (back into the BS if the current laser proves too finnicky or into another oplev (likely ITMy)).