TITLE: "10/28 [DAY Shift]: 15:00-23:00UTC (08:00-16:00 PDT), all times posted in UTC"
STATE Of H1: Lock Acquisition
SUPPORT: Jeff K., Kiwamu, Jeff B.
SHIFT SUMMARY: Not much happened during the day. The ifo stayed locked until toward the very end of the shift. Difficulty recovering from the lockloss due to many optics became misaligned. I restarted GraceDB script once today.
INCOMING OPERATOR: Patrick
ACTIVITY LOG:
15:25 Fil to Mid Y pick up EOM stuff
15:57 Fil back
19:04 Out of Observation to run DARM and Pcal open loop gain transfer function
20:13 Kyle to HAM3 after TF done. Cheryl to HAM12 area.
----------------Jeff Bartlett covering for me from 21:00-22:00----------------
21:33 Lockloss.
There're few other things I couldn't make sense off his notes. Will post more later when I do!
-------------------------------------------------------------------------------------------
22:00 I came back to find the ifo have lost lock. Jeff reported ifo was locking in green but couldn't move on because the power wasn't high enough. The ALSX beam spot have moved significantly. Almost every optics I looked at moved by significant amount (including ITMs and ETMs but not TMS) and I wasn't sure where I should start touching. Everyone was out for the 3 O'clock meeting. I didn't notice that IMC was locking at the wrong mode.
23:00 Patrick took over.
Ps. I might have missed a couple of activities (outside the LVEA, low impact) during the chaos. Sorry about that.
I stopped in DC readout and Evan and I went into the LVEA to look for the flashing light that Kyle reported (alog 22931). We think we may have located it somewhere under HAM4 but could not determine the source without crawling under the output beam tube. So we left it for later investigation. I also found a phone near the PSL enclosure with the line still connected. The power was unplugged but I unplugged the line from the wall as well. I also unplugged the extension cord from the wall that the phone had been unplugged from. We also heard what sounded like a drop of water falling in a cave. The light was on in the CER and we turned it off on the way out. We noted that some of the light from the CER leaks into the LVEA through an opening in the wall for a cable tray. We were in there from around 00:51 UTC to 01:14 UTC. Jim and Dave turned off the wireless access point that was turned on for Kyle. (alog 22931) I had to engage the ISS second loop by hand. I had to restart the GraceDB query script. Jenne ran the a2l script. I accepted in SDF the alignment offsets that Cheryl made to MC2. (alog 22933) (see first attached) I reverted the SDF differences for the H1:SUS-ITMY_L2_DAMP_MODE8 and H1:SUS-ITMX_L2_DAMP_MODE5 filter modules (see attached). Jenne and I claim this is alright because the gain on these is 0. Could someone tell us if they changed these and why? Unfortunately LLO went down from a power glitch just as we came back. We went into observing at 02:01 UTC. The range is around ~ 79 MPc.
Also note that a pump cart is running by the east door of HAM3.
It seems to me like somebody was making a filter for violin mode third harmonics. Dave also reported that sombody was making changes to the filter module the same day.
TITLE: 10/28 [EVE Shift]: 23:00-07:00 UTC (16:00-00:00 PDT), all times posted in UTC STATE Of H1: Unlocked OUTGOING OPERATOR: Nutsinee QUICK SUMMARY: Lights appear off in the LVEA, PSL enclosure, end X, end Y and mid X. I can not tell from the camera if they are off at mid Y. Winds are less than 20 mph. ISI blends are at 45 mHz. Earthquake seismic band is between 0.01 and 0.1 um/s. Microseism is between 0.1 and 0.7 um/s. Nutsinee was struggling with locking ALS when I came in. Kyle had been working on HAM3 and the mode cleaner was locked on an odd mode. Cheryl moved MC2 and I did an initial alignment. Now attempting to get to NLN.
Plot 1: HAM3 rotation in Z, MC2 ptich and yaw, PR2 pitch and yaw, OMC DC shoing lock loss.
HAM3 rotation in Z is the biggest rotation signal.
MC2 shifts -13urad in pitch, and -28urad in yaw
PR2shakes but comes back to within 1urad of where it started.
Plot 2: All IMC optics and Recovery
MC1 and MC3 see a change at lock loss, and then some motion at relocking the IMC, but are unchnged, which we knew since they're in HAM2.
MC2 needed to move +50urad in pitch and an additional -12urad in yaw to recover the IMC with good power at MC2 TRANS.
Overall MC2 was moved +37urad in pitch and -40urad in yaw.
The fact that MC1 and MC3 positions are unchanged after recovering the IMC shows that the change of MC2 alignment, while big, did moved MC2 to match the previous MC1 and MC3 alignments.
Plol 3: HAM3 HEPI and ISI signals that I believe show that their alignment didn't change, though I'm not completely confident that I got the right channels to show this.
Those are good SEI channels to look at. All but the HEPI Pringles are servo'd to DC so something would be wrong if they weren't the same. There is some hysterisis in the HEPI though as it is overconstrained. Those 'extra' dofs are the pringles and are not servo'd at DC hence the trend seen in the HP channel. This is expected to have very little impact on the final ISI table position.
Pump cart running near East Door of HAM3 overnight -> Will shut down and disconnect tomorrow (assuming no leaks) Also, I had Richard energize the wireless for the LVEA for my work and I assume that it is still on? - Don't know how to turn it off. I noticed an intense flashing light located somewhere beneath BSC1 or BSC2 when I was connecting my instrument air line below the OMC. I have never noticed this in the past.
LVEA WAP is now powered off
Likely due to Kyle working on HAM3 (literally). ALSX beam spot moved so much that's impossible to reacquire lock right now. An initial alignment might be necessary but I'm not sure if it's a good idea to do it when Kyle is possibly still climbing on the chamber. I will wait until he's out.
The comparison yesterday had very different ground motions so it bears repeating with similar GM time.
Attached is comparison from 4am Tuesday (reference traces) and 2am this morning. Below 10hz on X Y & Z there is nothing to mention. Between 10 & 100hz or something on X & Y, there are both more and less noisy bands not obviously due to the GM. The Z dof looks very comparable. On the rotational DOFs, there are a few areas of interest: RX 0.5hz; RY 0.1--0.5hz & 20hz; RZ 20--80hz. I'll repeat this tomorrow to see if these notables are repeatable.
JeffK & HughR
Here are TFs and coherences between the ground STSs near HAM2 and the HEPI floating L4Cs. These comparisons are from 3am Monday and Wednesday with pale reference traces from before Grouting Monday. The grouted data look mostly just like before grouting but there are a couple bands where things look better now: Y, 1 & 35hz. Again to be repeated.
J. Kissel, K. Izumi, N. Kijbunchoo
We've gathered new DARMOGLTF measurements to continue our long-term investigating on the slow evolution of the calbriation parameters. Based on what I see in DTT alone, the uncorrected loop gain and pcal to CAL-CS transfer function remains within 5% and 5 [deg] of the reference model. Very good! I attach screen shots of the raw measurements and a conlog of the relevant settings, but more detailed analysis to come.
The new DTT results live here:
/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O1/H1/Measurements/DARMOLGTFs/2015-10-28_H1_DARM_OLGTF_7to1200Hz.xml
/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O1/H1/Measurements/PCAL/2015-10-28_PCALY2DARMTF_7to1200Hz.xml
and have been exported to the following:
/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O1/H1/Measurements/DARMOLGTFs
1 / (1 + G) = 2015-10-28_H1_DARM_OLGTF_7to1200Hz_A_ETMYL3LOCKIN2_B_ETMYL3LOCKEXC_coh.txt
2015-10-28_H1_DARM_OLGTF_7to1200Hz_A_ETMYL3LOCKIN2_B_ETMYL3LOCKEXC_tf.txt
-G = 2015-10-28_H1_DARM_OLGTF_7to1200Hz_A_ETMYL3LOCKIN2_B_ETMYL3LOCKIN1_coh.txt
2015-10-28_H1_DARM_OLGTF_7to1200Hz_A_ETMYL3LOCKIN2_B_ETMYL3LOCKIN1_tf.txt
/ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/O1/H1/Measurements/PCAL/
C / (1 + G) = 2015-10-28_PCALY2DARMTF_7to1200Hz_A_PCALRX_B_DARMIN1_coh.txt
2015-10-28_PCALY2DARMTF_7to1200Hz_A_PCALRX_B_DARMIN1_tf.txt
Sudarshan, Darkhan, RickS
Using data from the SLM tool for 67 minutes just before the measurement of the Pcal2Darm TF, we calculated the TF for the 1 kHz and 3 kHz Pcal lines.
The plots attached below show:
1) The "kappas"
2) The measured TF, with coherence and the fractional uncertainty
3) The data for the 1 kHz line and comparison with TF values interpolated from the measured TF data
4) The data for the 3 kHz line
At 1 kHz, the transfer coefficient calculated from the SLM data differes from the interpolated coefficient from the meassured transfer function by 1.01% in amplutude and 0.15 deg. in phase.
The script and plots will be submitted to the SVN tomorrow and I will add a comment giving their location when they are.
Jonathan, Ryan, Jim, Carlos, Dave
we have configured and started the syslog-ng daemons on all the front end computers. This was done when H1 was out of observing mode. We have also configured SSHD on the front end computers to log any successful and unsuccessful login attempts.
The new Beckhoff SDF system was showing some channels as being different but the difference value is below the resolution in the OBSERVE.snap file. For these channels I have accepted them and wrote a new OBSERVE.snap file. Some channels were changing continuously, I have unmonitored these. The SDF is now running with zero diffs.
A WSU TC undergraduate group toured the site this morning. Arrival time at LSB = 9:30 AM. Departure time = 11:40 AM. Group size = ~15 adults. Vehicles at the LSB = ~5 passenger cars. The group was in the control room near 11:30 AM.
O1 days 32 to 40
model restarts logged for Tue 27/Oct/2015
2015_10_27 09:43 h1sysecatc1plc1sdf
2015_10_27 10:18 h1sysecatc1plc1sdf
2015_10_27 11:52 h1broadcast0
2015_10_27 11:52 h1dc0
2015_10_27 11:52 h1nds0
2015_10_27 11:52 h1nds1
2015_10_27 11:52 h1tw0
2015_10_27 11:52 h1tw1
2015_10_27 11:55 h1nds0
2015_10_27 11:55 h1nds1
2015_10_27 11:56 h1nds0
2015_10_27 11:57 h1nds0
2015_10_27 12:05 h1broadcast0
2015_10_27 12:05 h1dc0
2015_10_27 12:05 h1tw0
2015_10_27 12:05 h1tw1
2015_10_27 12:06 h1nds0
2015_10_27 12:06 h1nds1
2015_10_27 13:22 h1sysecatc1plc1sdf
2015_10_27 13:29 h1sysecatc1plc1sdf
2015_10_27 14:08 h1sysecatc1plc1sdf
2015_10_27 14:40 h1sysecatc1plc2sdf
2015_10_27 14:42 h1sysecatc1plc3sdf
2015_10_27 14:44 h1sysecatx1plc1sdf
2015_10_27 14:44 h1sysecatx1plc2sdf
2015_10_27 14:46 h1sysecatx1plc3sdf
2015_10_27 14:48 h1sysecaty1plc1sdf
2015_10_27 14:48 h1sysecaty1plc2sdf
2015_10_27 14:50 h1sysecaty1plc3sdf
Maintenance Tuesday. Roll out of new Beckhoff SDF system, showing up as pseudo front ends (marked in grey). DAQ restarted for new EDCU channels. Restart needed following NDS issues with Beckhoff SDF's high DCU ids.
model restarts logged for Mon 26/Oct/2015 No restarts reported.
model restarts logged for Sun 25/Oct/2015
2015_10_25 14:37 h1nds1
2015_10_25 19:00 h1nds1
2015_10_25 19:03 h1nds1
unexpected crash on h1nds1.
model restarts logged for Sat 24/Oct/2015 No restarts reported.
model restarts logged for Fri 23/Oct/2015 No restarts reported.
model restarts logged for Thu 22/Oct/2015 No restarts reported.
model restarts logged for Wed 21/Oct/2015 No restarts reported.
model restarts logged for Tue 20/Oct/2015
Power outage at 06:33. Full restart report attached for the full day. Maintenance day restarts not associated with power outage recovery are:
2015_10_20 12:30 h1calcs
2015_10_20 12:30 h1calex
model restarts logged for Mon 19/Oct/2015 No restarts reported.
TITLE: "10/28 [DAY Shift]: 15:00-23:00UTC (08:00-16:00 PDT), all times posted in UTC"
STATE Of H1: Obseving at 78 Mpc
OUTGOING OPERATOR: Ed
QUICK SUMMARY: The ifo has been locked almost 19 hours. MICH breathing continues. Wind below 5 mph. Useism reaching 0.5 um/s (90th percentile dashed line). Nominal seismic activity in eq band.
TITLE: Oct 28 OWL Shift 7:00-15:00UTC (00:00-08:00 PDT), all times posted in UTC
STATE Of H1: Observing
SUPPORT: N/A
LOCK DURATION: Entire shift
INCOMING OPERATOR: Nutsinee
END-OF-SHIFT SUMMARY:
IFO still locked at 78Mpc. Wind Calm. Sei and µSei remain the same. 2 more ETMY glitches. No further oscillations occurred. Curious that when LLO broke lock we also experienced a dip in BNS range.
SUS E_T_M_Y saturating (Oct 28 08:19:22 UTC)
SUS E_T_M_Y saturating (Oct 28 08:32:10 UTC)
SUS E_T_M_Y saturating (Oct 28 09:19:04 UTC)
SUS E_T_M_Y saturating (Oct 28 09:57:46 UTC)
SUS E_T_M_Y saturating (Oct 28 11:39:19 UTC)
SUS E_T_M_Y saturating (Oct 28 11:57:32 UTC)
MID-SHIFT SUMMARY:
ACTIVITY LOG:
07:34 Oscillations: duration ~ 5 minutes. May correlate with a high SNR LF glitch that I observed on DMT Omega.
07:48 MICH (live) on NUC4 FOMW seems to be “breathing” somewhere between 6Hz and 20Hz. Perhaps this is tied in with the previous log entry?
07:57 Oscillation: @08:04 IMC-F_OUTMON(tidal) grew to +/- 5 cts. ASC-DHARD_Y & P grew to +/- 2000cts. total duration is ~ 12minutes. Correlating(?) DMT Omega glitch shows up at approx the start time of the oscillation
10:00 Reset GraceDB External Notification script
13:25 DARM glitched between 60Hz and 300Hz
I had to restart GWIstat to clear a problem with the reported GEO600 status that seems to have been caused by a network interruption during the Tuesday maintenance period. For LHO and LLO, the reported duration of the current "OK+Intent" status (as of 5:53 PDT) will be incorrect until the status changes, because GWIstat keeps track of that time. By the way, there is now another status display page produced in parallel with GWIstat: https://ldas-jobs.ligo.caltech.edu/~gwistat/gwsnap.html . It's a cleaner layout, but doesn't have the links to summary pages.
MID-SHIFT SUMMARY: IFO locked and Observing, coincidently with Livingston, @~81.3Mpc. no change in EQ Sei bands. µSei has seen an increase to .6µ. Wind is calm. There’s been 4 ETMY saturations and two rogue Oscillations that were detailed in a previous aLog. I attempted to restart DTT RF45MHzAM on NUC5 to no avail.
Due to the violin mode problem on 10/25, Sheila has asked me to investigate when this mode really started to rung up. The first plot attached shows that the amplitude of 1008.45Hz were consistant the day before the power glitch and three hours before power glitch (the small difference you see is within the mode normal fluctuation range). The second plot shows the 1008.45 Hz got rung up by an order of magnitude during the first lock acquired after the power glitch just like others. Because this mode didn't have a damping filter at the time, ideally the amplitude should have stayed where it was. However, the final plot shows that the amplitude became worse as time progress while other modes were either stable or being damped until it caused the problem on October 25th. Could anything that happened during the power lost caused the mode to change its phase as it seems to be slowly rung up by ETMY MODE3 that's been existing since before O1? Note that this violin mode had never rung up before. The investigation continues.
To ensure that the 1008.45Hz line hasn't been slowly ringing up all this time, I've looked back at the asd amplitude of this mode until October 1st. First plot attached shows the amplitude/sqrt(Hz) versus frequency of this particular mode, one plot per day. The second plot shows log amplitude versus time. I only plotted one data point per day (10:00:00 UTC if data available, or any time where the BNS range was stable and the ifo was locked for at least an hour). The last data point is today (10/28 02:00:00 UTC). This mode has been fluctuating between 1e-22 and 1e-21 since the begining of the month (10/01) up until 10/20. You can see clearly that the amplitude begins to rise above its nominal on 10/21 after the power outage on 10/20 and continues to grow exponentially until it started to cause problems on 10/25. Indicates that the amplitude grow was causing by a positive feedback, which Sheila found it to be ETMY MODE3.
To conclude this study: This mode wasn't, and hasn't been ringing up before October 20th. Why it started to ring up after power outage is unclear. I can't think of anything else but something must have changed to cause this mode to change phase during the power outage.
Should we worry...?
Was there a significant temperature excursion during the power outage?
Yes.
I've attached the plot of average temperature in the VEAs. After the power outage LVEA average temperature had three big dips of about half degree. Average temperature at EY seems to fluxtuate more often and EX had couple of large drops.
Which turns out to be just a coincidence with the power outage according to John.
21:36 Evan and Borja to HAM6
21:40 Evan and Borja back
21:44 Cheryl Out of LVEA