cheryl.vorvick@LIGO.ORG - posted 05:02, Thursday 19 September 2019 (52028)
mid-shift update
Still locked, some minor EQs from Indonesia, largest was M6.2, H1 easily handled in ISI config WINDY.
Still locked, some minor EQs from Indonesia, largest was M6.2, H1 easily handled in ISI config WINDY.
TITLE: 09/19 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 114Mpc
OUTGOING OPERATOR: Ed
CURRENT ENVIRONMENT:
SEI_CONF state: WINDY
Wind: 12mph Gusts, 9mph 5min avg
Primary useism: 0.04 μm/s
Secondary useism: 0.25 μm/s
QUICK SUMMARY: locked in observe
TITLE: 09/19 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 113Mpc
INCOMING OPERATOR: Cheryl
SHIFT SUMMARY:
H1 locked and Observing for 3hrs 42minutes. Handing of to Cheryl
LOG:
Lockloss at 01:08:
Re-lock attempt -1
Re-lock attempt -2
Re-lock attempt -3
02:02 Initial Alignment
TITLE: 09/18 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Commissioning
OUTGOING OPERATOR: Niko
CURRENT ENVIRONMENT:
SEI_CONF state: WINDY
Wind: 9mph Gusts, 6mph 5min avg
Primary useism: 0.03 μm/s
Secondary useism: 0.24 μm/s
QUICK SUMMARY:
Some commissioning going on while Livingston was doing calibration and a power outage at LLO prompted a bit longer period for us.
TITLE: 09/18 Day Shift 15:00 – 23:00 (08:00-16:00), all times posted in UTC
STATE of H1: Commissioning
INCOMING OPERATOR: Ed
SHIFT SUMMARY: Fast lockloss in the early morning, recovered around noon. Went into commissioning at 1 in conjunction with LLO.
LOG:
15:00 (08:00) Start of shift
16:21 (09:21) Lockloss (SR2 M1 saturation prior)
16:57 (09:57) Tyler driving forklift around site
17:10 (10:10) Tyler finished
17:11 (10:11) Lockloss from ENGAGE_ASC_FOR_FULL_IFO
17:30 (10:30) Second lockloss from ENGAGE_ASC_FOR_FULL_IFO
17:34 (10:34) Starting initial alignment
17:53 (10:53) Initial alignment complete, starting re-lock
17:56 (10:56) Kyle CER -- grab squirt bottle
17:58 (10:58) Kyle out of CER
18:08 (11:08) Karen driving to woodshop
18:57 (11:57) At NLN, going into Observing
20:00 (13:00) Going out of Observing to start commissioning
23:00 (16:00) End of shift
I restarted the GDS calibration pipeline on the testing machine (l1dmt3) around GPS time 1252869000. The purpose of the restart was to test the wait time feature of gstlal-calibration-1.2.11. The new configuration file being used can be found in the calibration SVN revision 8459 here:
aligocalibration/trunk/Runs/O3/GDSFilters/H1GDS_1252867296_TEST.ini
The wait time is set to 10 seconds, so if there is no raw data available for 10 seconds, the pipeline will automatically start filling in this missing data with zeros.
I've done some testing to see how well the line subtraction algorithm in the gstlal calibration pipeline works when used to remove the roaming calibration line (> 1kHz, and changes periodically). Since the line subtraction has already been reviewed, I mainly wanted to investigate what happens right around the time that the line frequency changes. The gstlal calibration pipeline reads in the line frequency from the raw frames, so it is able to change frequency. Fortunately, the line frequency only changes when we are not in low noise, so there should be no concern about noise being injected at the moment of the change. The attached plots show data from around a time when the frequency changed from 2501.3 Hz to 2001.3 Hz, at GPS 1239997514 (Mon Apr 22 19:44:56 UTC 2019).
The first four plots are ASDs and ASD ratios from just before the change. Note that the 2501.3 Hz line is not subtracted very effectively.
The next four plots are ASDs and ASD ratios from just after the change. After the change, the 2001.3 Hz line is subtracted effectively, as expected.
The next (9th) plot is a time series of ratios of (cleaned data) / (uncleaned data) at the roaming line frequency, showing magnitude and phase. Note that the line subtraction of the 2501.3 Hz line stops working before the time of the change. It appears as though the algorithm is switching the frequency before the frequency actually changes. I will need to investigate this more to find out why this is happening. Hopefully, it can be fixed by a configuration change.
The last plot is similar to the previous one, except that it is for the regular calibration lines, so it provides a reference for roughly how effective the subtraction should be.
After some investigation, I found the problem causing the line frequency to be updated too early. When run offline (in DCS/C01 mode), the gstlal calibration pipeline has access to large amounts of data all at once, allowing computationally cheap processes to operate much faster than expensive ones. The reading in and updating of the calibration line frequencies is cheap, so it happens quickly, before the calibrated h(t) and subtraction algorithm are ready. The process of updating calibration lines involves changing a property of a gstreamer element, which is not necessarily timestamp-synced, unlike many other processes, such as adding two streams. The fix to the problem was to use the GstController, a tool which allows GObject properties to be updated over stream time. I've made this fix, it appears to have worked. The first attached plot is a time series of ratios of (cleaned data) / (uncleaned data) at the roaming line frequency, similar to the one in the above aLOG. The update occurs just before the 4-hour mark in the plot. The second plot is a similar plot showing all the other calibration lines, which do not change during this time.
Ops Shift Transition: 09/18/2019, Day Shift 15:00 – 23:00 (08:00-16:00) - UTC (PT)
State of H1: Locked
Intent Bit: Oberving
Weather: 0-5 mph wind
Primary 0.03 – 0.1Hz: 0.01 um/s, receding after EQ
Secondary 0.1 – 0.3Hz: 0.2 um/s
Outgoing Operator: Cheryl
Quick Summary: Locked and Observing for ~ 9 hours. Microseism up over the last ten hours.
TITLE: 09/18 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 115Mpc
INCOMING OPERATOR: Niko
SHIFT SUMMARY: locked all shift
LOG:
Locked in Observe, range is 114Mpc.
TITLE: 09/18 Owl Shift: 07:00-15:00 UTC (00:00-08:00 PST), all times posted in UTC
STATE of H1: Observing at 113Mpc
OUTGOING OPERATOR: Ed
CURRENT ENVIRONMENT:
SEI_CONF state: WINDY
Wind: 8mph Gusts, 6mph 5min avg
Primary useism: 0.02 μm/s
Secondary useism: 0.19 μm/s
QUICK SUMMARY: locked in Observe
TITLE: 09/18 Eve Shift: 23:00-07:00 UTC (16:00-00:00 PST), all times posted in UTC
STATE of H1: Observing at 114Mpc
INCOMING OPERATOR: Cheryl
SHIFT SUMMARY:
LOG:
This lock only held for 2 hours. I'm going to do Initial Alignment.
H1 back to Observing 5:55UTC - 1 hr 27 minutes door to door
H1 dropped out of lock after a long battle with high winds but the wind started to die back prior to so I didn't actually claim that as then reason. It took ~ 3 attempt to re-lock
WP8349
Betsy and I swapped the TCSY chiller with the rebuilt original chiller. This chiller was swapped back in January due to a leaky shaft seal. There was lots of corrosion in the pump, so we got a brand new pump and motor mounted and tested.
The hope with this chiller swap is that this chiller's temperature is a bit more stable and we won't have as many laser relocks that take us briefly out of observing (see alog 50789 for more info).
The laser recovered within an hour as usual, and temperatures and pressures all seem good. For the short time that we have been running with this current chiller, it looks good. Time will tell.
Current chiller: S/N127640801150617
Old chiller: S/N0110193301120813
-----
(Analog_1 is pressure in psi. Analog_2 I believe is actually temperature near or on the laser itself, not pressure as the medm states.)
Attachment 1 - The current chiller stabilizing.
Attachment 2 - One day trend, showing the current values compared to the old chiller.
A day later and things are looking a bit more stable than with the other chiller.
The "new" HEPI work covers fit on the HEPI housings, but because of the way the hoses are run, I think all of them will need to be reworked, or we'll have to re-route some of the hoses on all the housings. Hugh looked first thing and saw that one of the horizontal actuator hydraulic lines interferes with the cover. I took one to the warehouse and notched it, took it back out and installed it on the SE corner of HAM6. Didn't even trip the ISI or HEPI.
First image is the installed cover on the pier, second image is a close-up of the notch around the hose. Hard to see, but I made sure the line isn't touching the cover.
Third plot are asds for all of the HAM6 HEPI L4Cs, for both plots green (H3/V3) are the pier I put the cover on, brown (H4/V4) are the L4Cs on the pier on the opposite end of the blue cross-beam (according to page 40 of T1000388 , fourth image, lower left cartoon). I don't see any worrying differences, but there is a lot going on in the LVEA right now, I'll run another comparison tomorrow for a quiet time overnight.
Spectra from 2:41 local, I don't see any significant differences between the L4Cs. I'll poke around a bit more, but I don't see anything to worry about here.
02:17 Re-lock attempt -4
Re-lock attempt -5
03:17 H1 Observing
An hour spent trying to salvage a re lock.
1hour and 15 minutes - IA with relock (and ancillary aligning) <--- my new default approach.