edmond.merilh@LIGO.ORG - posted 08:16, Tuesday 28 March 2017 - last comment - 11:28, Tuesday 28 March 2017(35130)
BRSY Sensor Correction swithced OFF in prepeation for Maintenance
15:!5UTC
15:!5UTC
TITLE: 03/28 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 70Mpc
OUTGOING OPERATOR: Patrick
CURRENT ENVIRONMENT:
Wind: 11mph Gusts, 8mph 5min avg
Primary useism: 0.04 μm/s
Secondary useism: 0.40 μm/s
QUICK SUMMARY:
14:50 Saw Roto-Rooter coming on to site
14:55 Peter King goin into PSL enclosure
TITLE: 03/28 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC STATE of H1: Observing at 66Mpc INCOMING OPERATOR: Ed SHIFT SUMMARY: Observing entire shift. No issues to report. LOG: 09:51 UTC damped PI mode 27 by changing sign of gain Bubba will escort Apollo to mid Y and then survey mid X 14:26 UTC Chris to mid Y, use snorkel lift to work around roof edge 14:41 UTC Ken to mechanical room to install controller 14:49 UTC Rotorouter? through gate to see Bubba 14:56 UTC Hanford fire department through gate
See attached.
Temperatures are warming outside, enough so that I am comfortable with turning some of the heating elements of in the LVEA. In particular, Zone 2B has been turned back this morning. I will continue to monitor temps closely.
Have remained in observing. No issues to report.
TITLE: 03/28 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 65Mpc
OUTGOING OPERATOR: Nutsinee
CURRENT ENVIRONMENT:
Wind: 8mph Gusts, 7mph 5min avg
Primary useism: 0.04 μm/s
Secondary useism: 0.51 μm/s
QUICK SUMMARY:
No issues to report.
TITLE: 03/28 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 66Mpc
INCOMING OPERATOR: Patrick
SHIFT SUMMARY: One lockloss. No obvious environmental cause. Quickly recovered. Useism is creeping upward. TCS guardian kicked us out of Observe once. Not much else is going on.
LOG:
23:15 Gerado back
23:30 Balers done
01:58 CO2 laser lost lock (I think it was CO2Y). Back to Observe a minute later
06:04 Lockloss
06:40 Back to Observe
WP6543, FRS7701 Jonathan and Carlos:
the primary 2FA CDS login server cdsssh was upgraded from U12.04LTS to Debian 8. The reason for the upgrade is because U12 is end-of-life and will soon be unsupported.
TITLE: 03/27 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 65Mpc
OUTGOING OPERATOR: Jim
CURRENT ENVIRONMENT: Wind: 16mph Gusts, 13mph 5min avg Primary useism: 0.05 μm/s Secondary useism: 0.34 μm/s
QUICK SUMMARY: Been locked for 10.5 hours. Balers should be done by now (Bubba reported that they should be done by 16:30 local) Nothing else to report.
TITLE: 03/27 Day Shift: 15:00-23:00 UTC (08:00-16:00 PST), all times posted in UTC
STATE of H1: Observing at 67Mpc
INCOMING OPERATOR: Nutsinee
SHIFT SUMMARY:
LOG:
15:00 Bailers on Xarm
21:00 Betsy, TJ to Opstics Lab, out 22:00
21:15 Fil & Richard on Xarm, working on vault
22:00 Gerardo to optics lab
21:30 Evan & Miriam doing blip glitch measurements while LLO is down
WP 6544, ECR E1700107,
I have updated the following frontend models to implement the time-domain DCPD cross correlation technique.
omc.mdl (master model)h1omc.mdlCAL_CS_MASTER.mdlh1calcs.mdlThe h1omc and h1calcs models will be installed and restarted tomorrow during the maintenance period.
[The changes]
Here is a list of things that I newly added today in the OMC models (namely h1omc.mdl and omc.mdl). Also the attached are screenshots of relevant part of the models.
h1calcs.Here is a list of things I added today in the CAL CS models, e.g. h1calcs.mdl and CAL_CS_MASTER.mdl.
H(L)1:CAL-DELTAL_A and H(L)1:CAL_DELTAL_B).After these changes, I confirmed that they compile without an error. They are ready for tomorrow's installation. The models are checked into svn.
Evan, Miriam,
While L1 was having a small lock loss, we made a series of injections (with H1 in comissioning mode) in the H1:SUS-ETMY_L2_DRIVEALIGN_Y2L_EXC channel. The injections are single sine-gaussian pulses that simulate blip glitches (see https://ldas-jobs.ligo-wa.caltech.edu/~miriam.cabero/sine-gaussian.png ).
We started very quiet and slowly increased the amplitude, so that only the last 3 of the 9 injections appear in GDS-CALIB_STRAIN. The GPS times of the injections are:
1174684631
1174684680
1174684715
1174684745
1174684793
1174684854
1174684889 *
1174684945 *
1174685355 *
* Can be seen in CALIB_STRAIN
We will be repeating this kind of injections at opportunistic times (L1 not observing) in the next days, taking different blip morphologies, and different amplitudes.
Only the loudest of these saturated the noisemons, and it did so by hitting an analog limit at plus/minus 22000 counts. I projected the drive signal into noisemon counts and looked at the last three injections on the list. In the first two, the noisemon signal tracks the drive, and the subtraction of the two is just noise. In the last (loudest) injection, the noisemon hits an analog saturation at both plus and minus 22,000 counts leaving a huge glitch in the subtracted data. This is good because it suggests that the only important analog limit in the noisemon is this threshold. I don't have time to document it now, but I've tried the same with a set of loud detchar injections, which go up to hundreds of Hz, and I get the same behavior. So when the drive signal does not push the noisemon beyond 22,000 counts, we can trust the subtraction, and anything we see has entered the signal between the DAC and noisemon; it's a glitch in the electronics and not a result of the DARM loop. Attached are the three subtractions, noisemon minus projected drive signal.
Apollo is working at Mid-Y on the HVAC controls upgrade project. Both air handlers are down and will be for the night. This may cause a slight temperature rise in the building and may trigger some alarms. These are minor and only temporary.
Jim, Dave:
At 15:44:58 UTC (08:44:58 PDT) we received a timing error which only lasted for one second. The error was reported by the CNS-II independent GPS receivers at both end stations, they both went into the 'Waiting for GPS lock' error state at 15:44:58, stayed there for one second, and then went good. The IRIG-B signals from these receivers are being acquired by the DAQ (and monitored by GDS). The IRIG-B signals for the second prior, the second of the error, and the following two seconds (4 seconds in total) are shown below.
As can be seen, even though EX and EY both reported the error, only EX's IRIG-B is missing during the bad second.
The encoded seconds in the IRIG-B are shown in the table below. Note that the GPS signal does not have leap seconds applied, so GPS = UTC +18.
| Actual seconds | EX IRIG-b seconds | EY IRIG-b seconds |
| 15 | 15 | 15 |
| 16 | missing | 16 |
| 17 | 16 | 17 |
| 18 | 18 | 18 |
So EY was sequential through this period. EX slipped the 16 second by a second, skipped 17 and resynced at 18.
Summary: All problems were in CNS II GPS Channels at LHO. No problems were observed in the Trimble GPS Channels at either site, nor in the LLO CNS II Channels, with the exception of a change of -80ns in the LLO Trimble GPS PPSOFFSET a few seconds after the anomally (see below). It seems that both LHO CNS II Clocks simultaneously dropped from 10 to 3 satellites tracked for a single second. There is no channel recording the number of satellites locked by the Trimble clocks, but the RECEIVERMODEs at both sites remain at the highest level of quality, OverDeterminedClock (level 5 for the Trimbles) with no interruption at the time of the anomaly. It is unclear whether the LLO PPSOFFSET is causally related to the LHO event; the lack of other anomalous output from the LLO Trimble clock suggests that it is otherwise performing as intended. Descriptions of anomalous plots below. All anomalous plots are attached. Dilution of precision at BOTH LHO CNS II clocks skyrockets to 100 around the event (nominal values around 1) (H1:SYS-TIMING_X_GPS_A_DOP, H1:SYS-TIMING_Y_GPS_A_DOP). Number of satellites tracked by BOTH LHO CNS II clocks Plummets or two seconds from 10 to 3 (H1:SYS-TIMING_X_GPS_A_TRACKSATELLITES, H1:SYS-TIMING_Y_GPS_A_TRACKSATELLITES). In the second before the anomaly, Both of the LHO CNS II Clocks' RECEIVERMODEs went from 3DFix to 2DFix for exactly one second, as evidenced by a change in state from 6 to 5 in their channels' values (H1:SYS-TIMING_X_GPS_A_RECEIVERMODE, H1:SYS-TIMING_Y_GPS_A_RECEIVERMODE). The 3D Speed also spiked right around the anomaly for both LHO CNS clocks (H1:SYS-TIMING_X_GPS_A_SPEED3D, H1:SYS-TIMING_Y_GPS_A_SPEED3D). LHO CNS II Clock's 2D speeds both climb up to ~0.1 m/s (obviously fictitious) (H1:SYS-TIMING_X_GPS_A_SPEED2D, H1:SYS-TIMING_Y_GPS_A_SPEED2D). LHO Y-End CNS II Clock calculated a drop in elevation of 1.5m following the anomaly (obviously this is spurious) (H1:SYS-TIMING_Y_GPS_A_ALTITUDE). LHO X-End CNS II Clock thinks it dropped by 25m following the anomaly! I'm not sure why this is so much more extreme than the Y-End calculated drop (H1:SYS-TIMING_X_GPS_A_ALTITUDE). The Livingston Corner GPS PPSOFFSET went from its usual value of ~0+/-3ns to -80 ns for a single second at t_anomaly + 3s (L1:SYS-TIMING_C_GPS_A_PPSOFFSET). The GPS Error Flag for both LHO CNS II clocks came on, of course (H1:SYS-TIMING_Y_GPS_A_ERROR_FLAG, H1:SYS-TIMING_X_GPS_A_ERROR_FLAG)
Using my very limited knowledge of Windows administration, I have attempted to list the events logged on h1ecatc1 from 8:00 - 10:00 AM on Feb. 27 2017. Attached is a screenshoot of what was reported. I don't see anything at the time in question. However, there is a quite reasonable chance that there are other places to look that I am not aware of and/or I did not search correctly.
Starting CP3 fill. LLCV enabled. LLCV set to manual control. LLCV set to 50% open. Fill not completed after 3600 seconds. LLCV set back to 15.0% open. Starting CP4 fill. LLCV enabled. LLCV set to manual control. LLCV set to 70% open. Fill completed in 1127 seconds. TC A did not register fill. LLCV set back to 44.0% open.
Raised CP4 to 45% open.
Manually overfilled CP3 from control room at 100% open (@1:35pm local). Took 9 min. to overfill. Raised nominal value to 17% open.
About an hour after coming on shift I noticed something in the CR air and put on my mask, but after a while it was clear that that didn't help, so I started looking for other things, and realized it sort of smelled like burnt plastic. Called Corey, Richard, Dave, and a couple other people. Turned off most of the CR computers (about 4 warm ones), checked the MSR, checked the Computer user's room, and the smell was only in the CR. Tried to contact Robert, waited, but then turned off his computer, and am now waiting to see if that computer was overheating, and if turning it off clears up the air.
Currently watching H1 from the Computer Users Room, and have Verbal Alarms running in here.
we were not able to pin down where the odor was coming from, and there are no odors today.
That's 15:15UTC
15:20 UTC I also moved SUS ITMY to "SAFE" and took ITMY ISI to "OFFLINE" for CPS Noise investigations.
15:30UTC Due to impending PCAL work. BRSX is also off. Current state: "VERY_WINDY_NOBRSXY"
18:24 Hugh is finished his work and ITMY has been returned to it's nominal state(s)