Doubling these limits should virtually eliminate locklosses of the nature reported here.
The SDF has been updated, both OBSERVE and safe files; and they are committed to the svn.
Title: 12/1 OWL Shift: 08:00-16:00UTC (00:00-8:00PDT), all times posted in UTC
State of H1: Maintenance Tuesday
Shift Summary: Still cruising on a 28 hour lock. The DARM spectrum was getting a tiny bit noisy in frequencies < 50Hz for the last hour or two (that I noticed at least). Since LLO was down and the inpending maintenance day upon us, I didn't do look too far into it.
Incoming Operator: Jeff B
Activity Log:
Just checking on the reference cavity transmission, which continues to fall. The ambient temperature didn't change much. Doesn't appear to be due to variation in the pre-modecleaner transmission, nor temperature.
Starting some maintenance tasks since LLO has been out. We are currently still locked, but that won't last for long.
***** Incident Type -- Scheduled Outage Start Date and Time — 2015-12-15 9:00 CST (UTC-6) End Date and Time -- 2015-12-15 ~12:00 CST Service(s) Affected — This outage is required to move GraceDB onto new hardware. (GraceDB is currently running on temporary hardware after the unscheduled outage on Nov. 13th.) During the migration, GraceDB will be unavailable for a time period of ~hours. -- Please direct all questions to uwm-help@gravity.phys.uwm.edu
Observing at 73Mpc for 25 hours.
Environment calm. There have been a handful of glitches, but it doesn't seem like anything out of the ordinary.
Title: 12/1 OWL Shift: 08:00-16:00UTC (00:00-8:00PDT), all times posted in UTC
State of H1: Observing at 78Mpc for the last 20hrs
Outgoing Operator: Travis
Quick Summary: Travis had it easy, still locked from my shift yesterday. Wind minial, useism 0.4 um/s, all lights off, CW inj running, timing error on H1SUSETMY, IPC error on H1ISCEY (I'll clear them when I get the chance).
Title: 11/30 Eve Shift 0:00-8:00 UTC (16:00-24:00 PST). All times in UTC.
State of H1: Observing
Shift Summary: Very quiet shift. Only 6 ETMy saturations not related to any RF45 glitching. 20 hours of coincident observing with LLO.
Incoming operator: TJ
Activity log: None
Nothing of note. A few ETMy saturations not related to any RF45 glitching. Coincident observing with LLO just over 16 hours.
Laser Status:
SysStat is good
Front End power is 31.28W (should be around 30 W)
Frontend Watch is GREEN
HPO Watch is RED
PMC:
It has been locked 6.0 days, 7.0 hr 24.0 minutes (should be days/weeks)
Reflected power is 1.635Watts and PowerSum = 25.14Watts.
FSS:
It has been locked for 0.0 days 14.0 h and 16.0 min (should be days/weeks)
TPD[V] = 0.7883V (min 0.9V)
ISS:
The diffracted power is around 8.313% (should be 5-9%)
Last saturation event was 0.0 days 14.0 hours and 16.0 minutes ago (should be days/weeks)
At 12:58:31 PST the IOP for h1susey took 69uS for a single cycle which in turn caused a single IPC receive error on h1iscey. This TIM error has been occuring approx once a week for EY, in this case it is the IPC error which is unusual. We should clear these accumulated errors the next time we are not in observation mode or during Tuesday maintenance, whichever is the soonest.
TITLE: 11/30 DAY Shift: 16:00-00:00UTC (08:00-16:00PDT), all times posted in UTC
STATE of H1: Locked at 72Mpc for 12+hrs
Incoming Operator: Travis
Support: None needed
Quick Summary: Very quiet shift with H1 hovering between 75-80Mpc. 0.03-0.01 seismic is noticeably trending down over the last 24hrs. useism is holding just under 0.5um/s (so we still have 45mHz Blends ON).
Shift Log:
Last Tuesday (24th Nov) Jim and I modified the monit on h1hwinj1 machine such that when it restarts the psinject process it smoothly ramps the excitation amplitude over a time period of 10 seconds. We manually started the new system on Tuesday and since then there have been no crashes of psinject until the last 24 hours. There have been 4 stops (with subsequent automatic restarts) in the past 24 hours, each stop was logged as being due to the error:
SIStrAppend() error adding data to stream: Block time is already past
Here are the start and crash times (all times PST). Monit automatic restarts are maked with an asterix
| time of start | time of crash |
| Tue 11/24 14:55:47 | Sun 11/29 17:15:56 |
| Sun 11/29 17:16:00* | Mon 11/30 00:00:14 |
| Mon 11/30 00:01:13* | Mon 11/30 13:09:07 |
| Mon 11/30 13:09:36* | Mon 11/30 13:12:43 |
| Mon 11:30 13:13:39* | still running |
Adding INJ tag so the hardware injection team sees this.
Using a script written to flag CW injection transitions (start or stop) from psinject log files on h1hwinj, I found all of the transitions below to occur during observing mode. I'll ask Laura Nuttall to insert these in the database. 1132905630 1132905633 Nov 30 2015 08:00:13 UTC 1132905689 1132905692 Nov 30 2015 08:01:12 UTC 1132952963 1132952966 Nov 30 2015 21:09:06 UTC 1132952992 1132952995 Nov 30 2015 21:09:35 UTC 1132953179 1132953182 Nov 30 2015 21:12:42 UTC 1132953235 1132953238 Nov 30 2015 21:13:38 UTC Peter Shawhan reported via e-mail that these restarts happened sporadically in iLIGO days and may be related to a busy network. Presumably, these restarts were not disruptive in those days because they were not injected via a time-domain inverse actuation filter. Looking at the first of the crashes above (see attached time-domain zooms of the CW injection channel, the IAF-filtered version of it and DARM), I'm surprised that the effect in DARM isn't larger.
On Nov3, the ITMY Coil Driver was powered down in an attempt to clear its brain which had been giving false bad status indicators. We also changed the code so as to not drop out during these glitches, see T1500555.
Trending the channels show no status drop outs since the 3 Nov power cycle. Before the power cycle, the status had indicated a problem erroneously several times with at least twice dropping the IFO out of observing.
For quick reference, and if it wasn't made clear from the Primary Task, these are the ISI ST1 and ST2 coil drivers (not SUS coil drivers).
VerbalAlarm script was stopped, but unfortunately when trying to restart it, I get an error (see below). Not sure of a back-up alarm system to employ if this goes down. I have the Calibration medm (with GraceDB up on my work station), but not sure what else we should have up as a back up.
It's back. TJ called back and let me know of some missing parenthesis needed on Line 638.
No notable changes for first part of shift. Seismic is pretty much unchanged.
H1 is going on an 8+hr lock with a range which is hovering a little over 75Mpc.
Beamtube work has been canceled today due to a non-functioning manlift, and freezing conditions.
Attached are 7 day trends for all active H1 oplevs in pitch, yaw, and sum.
