Ran well over the weekend.
Went over maintenance items for tomorrow (see whiteboard for plans).
There are also a few commissioning, calibration, and PEM items that will require time with a locked IFO either tomorrow or over the week.
not sure of the cause yet. The 60-200Hz noise came back a few times before the lock loss, maybe related?
Adjusted the PSL Beckhoff computer time to daylight savings. The clock was about 10 minutes off in addition to the hour.
EX had a dip in pressure around March 9th from 97.7 to 97.5, but other than that all looks good.
I forgot to link to FAMIS4530
Closed
Looking over the past week, the daily EY gliches are at 20:44 PDT (03:44 UTC). On Saturdays a second glitch occurs at 21:04 PDT (04:04 UTC). Attached plots show 03:25 to 04:10 UTC for 3/19 and 3/12 (two successive saturday evenings in local time).
Here are data from End Y with most of the pressure channels and the controller output to the pump motor.
The attached plot gives support to this argument:
Something is happening with either the Pump motor or the VFD causing an increase in pressure out from the pump. This is evident by the increasing delay in pressure response as we proceed down the Pump Station Manifold. PRESS1 is the first after the Pump, the rest are after Laminar flow resistors and the filter. BSCSUP_PRESS is way down the line at the Chamber. The PRESS1 clearly leads the down stream pressures and it steps more sharply than the others. So this is a real pressure change increasingly softened by the accumulators and resistors etc as the wave moves down the line.
The VOUT signal is the PID loop responding to the differential pressure (diff between BSCSUP and BSCRET[not shown].) There is a big step down on this channel just before the pressure step up but that wouldn't cause an increase in pressure. So, I don't believe the motor is being directed to drive harder.
I'll look over the VFD manual to see if it has any scheduled items that might cause this? That just hardly makes sense...
Edit--Extra--If the output voltage of the PID controller could actually be manipulated without the VOUT data signal registering it, that too would be a possibility...Dave is looking closer at the computer.
TITLE: 03/20 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 67Mpc
OUTGOING OPERATOR: Corey
CURRENT ENVIRONMENT:
Wind: 12mph Gusts, 9mph 5min avg
Primary useism: 0.05 μm/s
Secondary useism: 0.16 μm/s
QUICK SUMMARY: A few min after I walked in there was some elevated noise in DARM from about 60-200Hz that lasted for a little less than 10min. The seismic BRLMS from 3-30Hz show a large amount of motion around then, but that may just be everyone arriving on site. The noise seems to have gone away for now, but I'll have to keep a close eye on it.
TITLE: 03/20 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 67Mpc
INCOMING OPERATOR: TJ
SHIFT SUMMARY:
Nice quiet shift with H1 locked over 38hrs. Winds are hovering around 10mph.
LOG:
Rebooted video4 in the middle of the night.
TITLE: 03/20 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 64Mpc
OUTGOING OPERATOR: Patrick
CURRENT ENVIRONMENT:
Wind: 11mph Gusts, 10mph 5min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.16 μm/s
Fairly quiet looking night so far. The winds look like they have been slightly inching up over last 3hrs (still at a calm ~10mph)
QUICK SUMMARY:
H1 currently locked over 30hrs.
TITLE: 03/19 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC STATE of H1: Observing at 68Mpc INCOMING OPERATOR: Corey SHIFT SUMMARY: PI mode 23 glitched a couple times. No other issues. Observing entire shift.
Have remained in observing. No changes in status or issues to report.
TITLE: 03/19 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 68Mpc
OUTGOING OPERATOR: Travis
CURRENT ENVIRONMENT:
Wind: 6mph Gusts, 5mph 5min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.18 μm/s
QUICK SUMMARY:
No issues to report.
TITLE: 03/19 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 67Mpc
INCOMING OPERATOR: Patrick
SHIFT SUMMARY: Observing for the entire shift and 22+ hours total for this lock. Rode through a 6.0 EQ in Solomon Islands early in the shift. No issues to report.
LOG: None
6.0M Auki, Solomon Islands
Was it reported by Terramon, USGS, SEISMON? Yes, Yes, No
Magnitude (according to Terramon, USGS, SEISMON): 6.0, 6.0, NA
Location: 68km N of Auki, Solomon Islands; LAT: -8.1, LON: 160.7
Starting time of event (ie. when BLRMS started to increase on DMT on the wall): 15:40 UTC
Lock status? H1 stayed locked, L1 broke lock.
EQ reported by Terramon BEFORE it actually arrived? Not sure
Observing for 18.5 hours. No issues thus far.
TITLE: 03/19 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 66Mpc
OUTGOING OPERATOR: Ed
CURRENT ENVIRONMENT:
Wind: 5mph Gusts, 3mph 5min avg
Primary useism: 0.01 μm/s
Secondary useism: 0.18 μm/s
QUICK SUMMARY: No issues were handed off.
TITLE: 03/19 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 66Mpc
INCOMING OPERATOR: Travis
SHIFT SUMMARY:
Quiet. Locked for 14hrs20min. H1 could use a2l but coincidental lock and no range loss has kept running it at bay.
LOG:
TITLE: 03/19 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 68Mpc
OUTGOING OPERATOR: Patrick
CURRENT ENVIRONMENT:
Wind: 15mph Gusts, 12mph 5min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.20 μm/s
QUICK SUMMARY:
TITLE: 03/18 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC STATE of H1: Observing at 67Mpc INCOMING OPERATOR: Ed SHIFT SUMMARY: Winds have subsided to below 20 mph. No major issues after going back to observing. SEI is back to its nominal state (Jim W. was here after Travis lost lock due to the wind). If the winds come back up Jim W. has suggested trying the newly created 'MORE_WINDY' SEI state (see his alog). LOG: 23:29 UTC Starting relocking. 23:51 UTC Took about four tries to get past CHECK_IR. 00:00 UTC Lock loss from SWITCH_TO_QPDS. 00:23 UTC Stopping at DC_READOUT_TRANSITION. Jim W. ramping off sensor correction. 00:25 UTC Jim W. switching SEI back to nominal. 00:39 UTC Observing. 01:02 UTC Jim W. leaving. Changed sign of gain to damp PI mode 28. 01:08 UTC Changed phase to damp PI mode 27. 06:46 UTC The bottom monitor connected to nuc6 lost its signal for ~ 20 sec.
The winds have come back down and we are back to observing.
The short version: Wind (finally, at 50 mph) gave us trouble locking today. I tried some stuff that may have helped, but I need commissioning time (and wind!) to see if it will do any good, but ALS REFL CTRL seems a good IFO witness. Everything is switched back after winds died down.
Travis started have troubles locking when we were having sustained winds around 40 mph and gusts over 50, so I came out to see if some of the stuff we've tried in the past would help. I can't really say for sure if what I've done has helped because winds have now died down, but I wanted to log some ideas. As I found in alog 30547, it would probably be beneficial to push the St1 RX/RY blends down to reduce the amount that the BSCs tilt above .1 hz, so I switched the ST1 RX/RY blends from the normal Quite_250 to the Quite_90 blends. I have also worked on commissioning St2 RX/RY loops (which we don't normally use) that improve the St2 motion at 1-10hz so I tried turning those on. We also use some sensor correction from St1 to St2 around .5hz, which doesn't hurt us when winds are below 30mph, but could couple St1 tilt to St2 motion with high winds, so I tried turning this off. The first 2 attached plots show the EY ISI St2 seismometers and the ground seismometer (which is a better indicator ground tilt than Y motion at these wind speeds).
The first plot shows the St2 RX/RY GS13s and you can see the the table is moving less between .1-20hz (blue is the "high wind" state, red is nominal), while the wind had dropped off a little (cyan to pink).
The second plot shows that this improvement in RX didn't make a drastic difference in Y (green is the "high wind" state, brown is nominal), but we gave up a bit of isolation around .5 hz by turning off the St1-St2 sensor correction.
The third plot shows the difference in the ST2 CPS. Below the blend (250mhz) the CPS are a good witness of how much tilt is being injected into the table motion, so at 40 mph it would probably be a good idea to turn this off, if we can lock with more .5hz motion.
On Monday or Tuesday, I'll try to get a configuration (lower ST1 RX/RY blends, ST1-2 sc off, maybe a hack to get ST2 RX/RY iso loops on) in the SEI_CONF guardian.
I have added a state (and tested the transition while at DC READOUT) to the SEI_CONF called MORE_WINDY that turns off the ST1-2 sensor correction. Operators should try this state (MORE_WINDY) with high winds (40+) and low microseism.
These are winds and the range for the last 10 hours, note that the range started suffering when winds hit 30 mph at the CS and EY. We didn't lose lock until we had an almost 60mph gust at EX.
Back to observing after getting to low noise and losing lock to an unheard PI, and then a brief period of comissioning when we were back up again.