TITLE: Sep 17 Owl Shift 07:00-15:00UTC (00:00-08:00 PDT), all times posted in UTC
STATE Of H1: Lock Acquisition
OUTGOING OPERATOR: Ed M.
QUICK SUMMARY: Attempting to lock the interferometer. I couldn't get any flashes on POP 18 and 90 while trying to lock PRMI. Since Ed did the initial alignment when the ground motion was still high I think I might try to do the initial alignment again when the ground motion become less active.
TITLE: Sep 16 EVE Shift 23:00-07:00UTC (16:00-00:00 PDT), all times posted in UTC
STATE Of H1: Down
LOCK DURATION: N/A
SUPPORT: Hugh, Jeff K, Sheila
INCOMING OPERATOR: Nutsinee
End-of-Shift Summary: Recovering from Chilean Earthquake(s) barrage is an understatement as there have been close to 20 after-shocks (and a smattering of New Guinea and Alaska to round out the competition), most of significant magnitude. I was able to complete an initial alignment at, around, 03:10UTC and began the locking sequence but never made it past CHECK_IR. At that point the seismograph started to climb again. Fortunately there was no further tripping of systems. Mother Nature is really letting her hair down! ( and having a few). Handing off to Nutsinee.
Activity log:
23:00 IFO locked
23:23 LOCKLOSS Chilean Earthquake 8.4MAG 23:45UTC
23:30 Began damping Seismic systems with Hugh’s tuteledge
00:32 Jeff Kissel taking advantage of the earthquake break to do some charge measurements
00:45 Darkhan also doing some open loop measurements in this down time.
02:05 Attempt to bring back End Station ISIs seems succesful. GS 13s back to high gain. Seismic graph is showing 3µ/S.
02:09 Ditto for ITMs
02:10 Begin bringing up Input and Output HAM SEI.
02:15 Kissel brought ESDs back and restored ETM alignment.
02:31 All SEI back to nominal isolated states. GS13 gains switched back to HIGH.
02:34 IMC re-locked. Looks like the ground motion is still too great to begin aligning arms.
03:10 started initial alignment. Waited at DarkMich to watch excessive ground motion
03:34 GRB alarm. https://gracedb.ligo.org/events/view/E184908 .
03:58 began locking sequence. ot as far as CHECK_IR...too much ground motion and then more earthquakes.
04:00 switched Observation mode to Environmental
05:38 Ground motion starting to drop below 1µm/s. Attempt at locking DRMI shows way too much motion.
A little late with this entry due to Mother Nature:
TITLE: Sep 16 EVE Shift 23:00-07:00UTC (16:00-00:00 PDT), all times posted in UTC
STATE Of H1: Observing
OUTGOING OPERATOR: Patrick
QUICK SUMMARY:Full control room.. Wind is blowing just under the 20mph mark. Seismic activity quieting down from earlier earthquakes. All lights at Mid, End and LVEA stations are off. Then...Chilean earthquake 8.3Mag caused lockloss. Working with Hugh to get tripped seismic systems to happy places to ride this out.
Terramon hadn't reported the arrival of this event until it had already "knocked us for a loop".
C. Cahillane The five sensing function measurements and associated uncertainties at LHO have been analysed. Plot 1 shows the five measurements and models plotted together, alongside the residuals. The measurements are from August 26, 28, 29, and September 8 and 10. Plot 2 shows the weighted mean of the five measurement residuals in black, a naive systematic fit in red, and the weighted mean / systematic fit in blue. To be consistent with what I've done for actuation, right now I'm going to quadratically sum the systematic and statistical uncertainty, and zeroth order extrapolate it. For the sensing function though this is very clearly the incorrect approach, since the systematics on low freq sensing mag or high freq sensing phase are not simply going to stop blowing up at the end of the measurement. I will certainly have to go back and fix this. Also, since the systematics are so large here, I am thinking about applying my own "correction" to the sensing function the model gives me and then plotting only statistical uncertainty. I think this plot will display the directionality of our error alongside the statistical uncertainty. Things to do: 1) Calculate weighted mean of actuation residuals for each measurement time for Sudarshan 2) Zeroth Order -> First Order Extrapolation 3) Find systematics of actuation stages 4) Apply correct A_pu uncertainty calculations. 5) Apply systematics "corrections" to my uncertainty models to get directionality of error combined with statistical uncertainty 6) Go to LLO and do it all again!
J. Kissel, with help from B. Weaver Remotely While the earth was still cruising at 10 [um/s], ringing down from Giant EQ in Chile, I gathered some charge measurements on both ETMX and ETMY. I'm unsure of the data quality, because we were only able to get the SEI system to DAMPED (HPI on position loops, ISIs damped only with low-gained GS13s), but the results live here: /ligo/svncommon/SusSVN/sus/trunk/QUAD/H1/ETMX/SAGL3/Data/ data_2015-09-17-00-37-40 data_2015-09-17-00-50-55 data_2015-09-17-01-04-09 data_2015-09-17-01-17-51 data_2015-09-17-01-31-53 /ligo/svncommon/SusSVN/sus/trunk/QUAD/H1/ETMY/SAGL3/Data/ data_2015-09-17-00-37-41 data_2015-09-17-00-43-23 data_2015-09-17-00-56-34 data_2015-09-17-01-09-50 data_2015-09-17-01-24-05 data_2015-09-17-01-37-59 data_2015-09-17-01-39-45 data_2015-09-17-01-52-34 Note after the charge measurements were complete, I used the SDF system and conlog to restore the ESD Driver / Bias / Linearization settings and the alignment offsets, respectively. Will post results tomorrow.
RickS, DarkhanT, JeffK, CraigC, SudarshanK
After discovering a discrepancy in phase, of about 137 degrees, between measurement and model for the reference value of kappa_tst for LHO, we applied this phase as a correction factor and computed the kappas. Most of the parameters are close to their nominal values after applying this factor except for imaginary part of kappa_tst which is 0.05 (nominal 0) and real part of kappa_pu is about 8% off from its nominal value of 1. Cavity pole is also off by about 20 Hz from its nominal value of 341 Hz.
A similar correction factor (a pahse of 225 degrees) was applied to LLO refernece value of kappa_tst as well. For LLO, the imaginary part of kappa_tst and kappa_pu are close to zero wheras the real part are off by about 7% for kappa_pu and 10% for kappa_tst. kappa_C is close to 1 (off by few percent on some lock stretch) and Cavity pole is off by about 10 Hz from its nominal value of 388 Hz.
The 0 (zero) on X-axis of each plot mark the time when the most recent calibration data was taken for each observatories. For LLO it is: 14-Sep 2015 07:23:32 UTC and for LLO it is 10 Sep 2015 23:35:32 UTC.
Correction on that last entry. We bumped the fan at ~1545 PT.
STATE Of H1: Out of lock, riding out earthquake SUPPORT: Sheila, Jenne SHIFT SUMMARY: After Sheila and Jenne diagnosed the lock loss from the DSOFT loop, we were able to get back to observing without trouble. Took out of observing briefly to allow Sheila and TJ to load guardian code changes. Just lost lock to an earthquake in Chili. INCOMING OPERATOR: Ed ACTIVITY LOG: 21:51 UTC Sheila starting WP 5502, TJ starting WP 5499. 22:08 UTC Out of Observing for Sheila and TJ load guardian changes. 22:09 UTC Back to observing. 22:37 UTC Jeff B. to area in mechanical building near HEPI pump mezzanine to retrieve small bin of parts. 22:43 UTC Jeff B. back. 23:13 UTC lost lock to a large earthquake in Chili.
I turned off corner station SF-3 @ ~1350 PT. This fan has been suspect for some time (see FRS 3263) and while showing Tim Nelson from LLO our supply fans, there was what appears to be a bearing failure by the excessive noise and vibration. I described this to John W. at the All Hands Safety meeting and afterwards ~1545 PT we went and bumped the fan a couple of times and heard the noise and felt the vibration of the fan. I will put a work permit in to disassemble this fan and repair as soon as I am able to work on it. The fan is locked out. For more information see CSFanStatus_3.
While Sheila was making her Guardian changes, I got the OK to remove the BRS tests and add 2 others to DIAG_MAIN via WP 5499.
Tests removed:
Tests added:
The code was tested, then saved and loaded into the node. I have not committed this to the svn yet as we are still locked.
Svn'd after the earthquake got us.
Not sure why anything would prevent you from committing to the SVN....
Attached is agallery of 5 "dust" glitches. Still clueless of what they are, but - ETMY saturation is a symptom, not a cause - it is not possible to produce such a white glitch from saturating a drive. - The DCPD spectrum shows a roll-off for all of them - But the roll-off frequency (i.e. glitch duration) varies significantly = from about 300Hz to 3kHz. Example 2: GPS: 1126294545 UTC: Sep 14 2015 19:35:28 UTC ETMY saturation: yes Example 3 GPS: 1126437892 UTC: Sep 16 2015 11:24:35 UTC ETMY saturation: yes Example 4 GPS: 1126434798 UTC: Sep 16 2015 10:33:01 UTC ETMY saturation: yes Example 5 GPS: 1126441165 UTC: Sep 16 2015 12:19:08 UTC ETMY saturation: yes Example 6 GPS: 1126442379 UTC: Sep 16 2015 12:39:22 UTC ETMY saturation: yes
WIth Hang's help, I managed to investigate these glitches with the new lockloss tool using SUS-ETMY_L3_MASTER_OUT_LL_DQ as a reference channel. The script couldn't find any other optics that glitch prior to the ETMY. And sometimes the glitches are seen by ETMX 30-40 miliseconds after.
I've attached the plot of the glitches at the time you've given. I've also attached the list of channel I told the script to look. Basically all the SUS MASTER OUT DQ channels. Please let me know if you have any suggestions on whereelse I should look at.
Attached are time traces of the DCPD_SUM for the 5 examples.
We just loaded two minor guardian changes that should save us some time and locklosses during acquisition. They don't change anything about the configuration of the IFO in observing mode.
The first was the fix the issue that Jenne and Patrick wrote about 21508. When the SOFT loops had not converged before the gain was increased, it could cause us to loose lock (which could be because the SRC1 loop is strongly contaminated by these signals, and we see POP90 running away in the usual way.) I've simply added another if statement to the ENGAGE_ASC part 3 state, which will wait for all the SOFT loop control signals to become less than 500:
The Initial Alignment Checklist and ops wiki page's "Initial alignment brief version" have been updated to reflect this change.
When you compare "H1 SNSW EFFECTIVE RANGE (MPC) (TSeries)" data in DMT SenseMonitor_CAL_H1 with its copy in EPICS (H1:CDS-SENSEMON_CAL_SNSW_EFFECTIVE_RANGE_MPC), you will find that the EPICS data is "delayed" from the DMT data by about 109 seconds (109.375 sec in this example, I don't know if it varies with time significantly).
In the attached, vertical lines are minute markers where GPS second is divisible by 60. Bottom is the DMT trend, top is its EPICS copy. In the second attachment you see that this results in the minute trend of this EPICS range data becoming a mixture of DMT trend from 1 minute and 2 minutes ago.
This is harmless most of the time, but if you want to see if e.g. a particular glitch caused the inspiral range to drop, you need to do either a mental math or a real math.
(Out of this 109 seconds, 60 should come from the fact that DMT takes 60 seconds of data to calculate one data point and puts the start time of this 1 min window as the time stamp. Note that this start time is always at the minute boundary where GPS second is divisible by 60. Remaining 49 seconds should be the sum of various latencies on DMT end as well as on the copying mechanism.)
The 109s delay is a little higher than expected, but not to strange. I'm not sure where DMT marks the time, as the start/mid/end of the minute it outputs.
Start Time | Max End Time | Stage |
0 | 60 | Data being calculated in the DMT. |
60 | 90 | The DMT to EPICS IOC queries the DMT every 30s. |
90 | 91 |
The EDCU should sample it at 16Hz and send to the frame writter. |
The 30s sample rate of the DMT to EPICS IOC is configurable, but was chosen as a good sample rate for a data source that produces data every 60 seconds.
It should also be noted that at least at LHO we do not make an effort to coordinate the sampling time (as far as which seconds in the minute) that happen with the DMT. So the actual delay time may change if the IOC gets restarted.
EDITED TO ADD:
Also, for this channel we record the GPS time that DMT asserts is associated with each sample. That way you should be able to get the offset.
The value is available in H1:CDS-SENSMON_CAL_SNSW_EFFECTIVE_RANGE_MPC_GPS
20:28 UTC Back to observing mode.
Nairwita Mazumder, Rich Abbott A few days back Jim noticed (alog ) that the "Bumbling line" which varies over a large frequency range is again back on ETMX seismic channels . This was first noticed on March and disappeared before ER7 and again was seen from 4th August. One can see the lines at all the horizontal and vertical sensors on ETMX. I have attached a pdf containing some follow up work done during Rich's recent visit to LHO. The first plot in the pdf is the spectrogram of ETMX GS13 on 26th August. It can be seen that there are multiple wandering lines having a fixed offset. We were suspecting that some magnetometers at the End X might be the culprit (as we could not find any correlation between temperature fluctuation with the line ). The second and third plots are the spectrum of H1:PEM-EX_MAG_EBAY_SEIRACK_Z_DQ and H1:ISI-ETMX_ST2_BLND_Z_GS13_CUR_IN1_DQ for 2nd August and 26th August respectively. The red one is for 2nd August when the bumbling line could not be found and the blue one is the recent data (26th August). It is clear that the peaks appearing on ISI-ETMX_ST2_BLND_Z_GS13 after 3rd August are correlated with the peaks of the spectrum (which also appeared around the same time) of SEIRACK magnetometer . The plots on the second page shows the coherence between GS13 and the magnetometers in the VEA and SEIRACK. It looks like the magnetometer on the SEI rack has stronger coherence with GS13 sensors than the magnetometer located at VEA . I have marked two points (blue and red cross) in the coherence plots to highlight two of the many peaks.
Adding to Nairwita's comments, the signal seen in the GS13 spectra is also present in the magnetometer data. This being the case, it's most likely that the harmonic series points to an electromagnetic artifact associated with the HEPI pump variable frequency drive. The fact that the same signature does not exist at the other end station (I assume this to be true, but have not verified) may point to an enhanced susceptibility in the X-end electronics for some reason. No reason to panic much yet, but duly noted.
I have attached the coherence plots computed between PEM-EX_MAG_SEIRACK and GS13 , ST1 CPS and ST2 CPS over the frequency range 0.4Hz-900Hz to check the following two points: (1) If there exists any coherence between CPS and the Magnetometer at frequency above 256 Hz (2) What the low frequency behavior is I can be seen that the coherence between CPS and the magnetometer above ~25Hz is pretty low compared to GS13, but them have relatively high coherence with PEM-EX_MAG_SEIRACK near 20Hz .
I finished the initial alignment. After having trouble with INPUT_ALIGN I looked at IM4 and PR2 alignment. IM4 alignment was way off from the last time the ifo was locked. I paused the Guardian and brought IM4 back to where it was. After that it locked in the wrong mode so I adjusted PR2 YAW. I had a little trouble with MICH_DARK_LOCK. BS ISI Watchdog tripped twice. I requested DOWN and waited until the optic settled then tried again. I also had to touch PR3 to maximize COMM beatnote. Everything went well. Now begin lock aquisition. The ground motion has come back to its nominal after series of earthquakes.
Guardian stalled at DC_READOUT_TRANSITION. OMC locked at a wrong mode. Lost lock at DC_READOUT shortly. T2, T3, LF and RT osems had huge number of counts. I called Sheila and she suggested I monitor where the offsets came from. Now relocking.