I ran BruCo on one hour of data from last night. The report is available here:
https://ldas-jobs.ligo.caltech.edu/~gabriele.vajente/bruco_1137640217/
At low frequency, SRCL and DHARD_P are the largest contributions. SRCL is still a factor of few below the measured noise, while DHARD is very close around 14 Hz.
There is coherence with magnetometers (still far from the measured noise)
No significant broadband coherence in the mistery noise region.
A 7.1 earthquake in Alaska at 10:30 UTC tripped many watchdogs. I reset all and went back to nominal working conditions for all seismic isolation at about 17:30 UTC.
Aborting a DTT measurement caused a lockloss - no good, although not a new phenomenon.
Since there are so many SDF diffs, I'm wouldn't put the IFO in Observe anyway, so I'm going to just leave it Down for the night.
Reported as FRS Ticket 4274, so we begin to quantify how often and how painfully this remaining DTT abort problem affects us.
Evan, Gabriele
There are too many SDF to accept, so we're just leaving the IFO undisturbed in low noise, starting 3:10 UTC
I just got here to do some ASC stuff, so the IFO will no longer be undisturbed, as of 4:10 UTC.
We noticed a small bump in DARM at about 360 Hz, coherent with PRCL. We added a 270-430 Hz elliptic band-stop. DARM improved and the coherence is almost gone. We are losing 9 degrees of phase at 50 Hz (PRCL UGF).
The filter is implemented in the SUS filter bank.
The bandstop filter is now engaged by the guardian in the NOISE_TUNING step.
1220 - 1235 hrs. local -> To and from Y-mid Next manual overfill expected Monday, Jan 25th before 4:00 pm
First a bit of history: h1boot-backup is an identical machine to h1boot and can be used to replace h1boot if it were to fail. The /opt filesystem on h1boot used to be rsynced every hour to keep h1boot-backup current. Over the course of summer 2015 the rsync took longer, and became linked to the EPICS freeze ups of the front end computers. We suspect the main reason is that the NDS jobs directory became filled with many small files. Over Christmas I was doing select rsyncs manually, skipping these huge directories of NDS logs.
Over the last week I have reduced the number of files in these directories to a manageable number. This took some time because when a directory has 6.5 million files in it standard commands like find
or ls
wont work. I used customized C code to scan the files.
We now have NDS logs which look back over the past 7 days. The full rsync now only takes 15 minutes (it was taking either 10 hours or would not complete).
Because there is a non zero chance these fast rsyncs could freeze EPICS, I have scheduled the backups 3 times daily at 8am, noon and 6pm. If anyone sees epics freeze up at these times this would be the reason.
no restarts reported for both days.
Apologies for no transition entry. I was tasked with locking as soon as I walked in.
TITLE: Jan 22 EVE Shift 00:00-07:11UTC (16:00-23:11:00 PDT), all times posted in UTC
STATE Of H1: Down
SUPPORT: Sheila,Evan, Gabriele
INCOMING OPERATOR: N/A
ACTIVITY LOG:
00:28 HFD on site as a matter of protocol for a faulty fire alarm panel
00:30 another fire unit on site
00:42 reloaded ISC_LOCK Guardian
00:45 HFD Off site. Three firetrucks drove past corner station during lock
00:46 Dave B called and informed me of an AC failure. He’s contacting the proper folks.
01:13 Dick out of optics lab
01:32 Bubba back on site to handle AC issue in H2 building
01:37 BS ISI St2 watchdog trip
01:38 cleared HAM6 ACT watchdog accumulation counter
02:18 GRB alert
02:51 Bubba has reported there’s nothing he can do about the AC in the H2 building tonight so he’s leaving it off. He also went into the CER to check on the AC in there. He didn’t think anyone had been in there to check since the power outage. It seemed ok.
04:xx Relocked at NLN. Gabriele has turned me loose into the world!
We have a cooling problem with the H2 DTS building. Bubba came back in this evening to take a look at this, in the mean time we have powered down all DTS computers and DC power supplies for the weekend.
After an extensive search for the meaning of the error code displayed on both thermostats indicating a low pressure fault, which in and of itself is odd for both systems to have the exact same fault, I believe I have narrowed the problem down to faulty thermistors. This may have been caused by the recent power outage or may be simply coincidence. Parts will need to be ordered next week. I will follow up with a FRS.
I retuned the MICH feed-forward path. I injected noise in SRCL and measured the transfer function to DARM: TF_SRCL_DARM. Then I injected noise at the feed-forward input, with the FF shaping filters off, but with SBVio1 and SBVio2 on. I measured the transfer function from SRCLFF_IN2 to DARM: TF_SRCLFF_DARM. The optimal feed-forward filter should then be -TF_SRCL_DARM/TF_SRCL_FF. The first plot shows the measurement and the fit. Again, I decided not to fit the sharp features due to the bounce and roll filters.
I implemented the new filter in the bank 'newFF' and engaged it. The performance improved, as shown by the reduced coherence. The peak at 3 Hz also reduced a bit. See the second plot: green no feed-forward; blue old feed-forward, red new filter. The last plot compares the old (blue) and new (red) filters. With the new filter, I also switched on an AC coupling (double zero, double pole at 0.1 Hz).
Guardian code modified to load the new MICH and SRCL feedforward filters.
Sheila, Evan, Gabriele
We retuned the MICH feed-forward path. We injected noise in MICH and measured the transfer function to DARM: TF_MICH_DARM. Then we injected noise at the feed-forward input, with the FF shaping filters off, but with all the stop-band filters on and with the invBS filter on. We measured the transfer function from MICHFF_IN2 to DARM: TF_MICHFF_DARM. The optimal feed-forward filter should then be -TF_MICH_DARM/TF_MICH_FF. The first plot shows the measurement and the fit. We decided not to fit the sharp features due to the BS bounce and roll filters.
We implemented the new filter in the bank 'newFF' and engaged it. The performance improved, as shown by the improved DARM noise and reduced coherence. See the second plot: green no feed-forward; blue old feed-forward, red new filter. The last plot compares the old (red) and new (blue) filters. With the new filter, we also switched on an AC coupling (double zero, double pole at 0.1 Hz).
Reconfigure x1fs0 to export individual directories under /opt, rearrange nfs mounts for front ends, x1boot, and x1 DAQ computers. This is to test proposed changes for the h1 system.
SHIFT SUMMARY: Some locking success in the morning, but trouble in the afternoon with high winds and microseism. Jim W. tried putting the end stations on Quite_90 and enabling sensor correction. Ed took over locking attempts in the mid afternoon. The winds have come back down. ACTIVITY LOG: 19:59 UTC Jeff B. and Nicole W. to LVEA to look at 3IFO baffles 20:06 UTC Stopping locking attempts while Jim W. runs transfer functions on ETMX 20:36 UTC Jeff B. and Nicole W. done 20:48 UTC Phone call from Hanford Emergency instructing personnel in 200 West to take cover 21:38 UTC Phone call from Hanford Emergency reporting the lifting of the take cover alarm 21:44 UTC Dick G. to optics lab 21:45 UTC Resuming locking attempts. Jim W. has put the end stations on Quite_90 and turned on sensor correction. 23:35 UTC Gerardo to mid X to retrieve empty tote 23:54 UTC Filiberto to mid X to check alarm on fire panel Tour group in the control room around 19:50 UTC Dave B. changed the vacuum overview medm on video1
Fixed bug: Previously, when Verbal could not connect to a channel through ezca, it would crash with an EzcaConnectError. Now it will make a small noise and write a notification that it cannot connect to the channel and repeat. This will begin to get very useful now that we are in the commission era.
Saturations will now just say the optic name.
I also removed the acknowledgement feature for the CW inj. It should still alert if the inj status changes, but operators do not have to acknowledge.
uSeism is approaching 1um/s and the winds have dies down to below 10mph.
I hadn't been informed about the sensor correction/Quite_90 situation when I was asked to lock the interferometer upon my arrival. We have ben locking steadily on the 45mHz blends all evening. I've just changed everything back to Quite_90 to have a look. OK, it looks good so far. Apologies, Jim there was a lack of communication.
WP5692 FRS4202
John, Kye, Gerardo, Dave:
I have modified the H0 vacuum overview screen to show an alarm border around bar monitor widgets. If the widet has no content, the alarm color is now visible in the border.
Thanks Dave! "no content", "no valid content", "value not valid for parameter represented", "something is broken" etc...