Both L1 and H1 lost lock at times that coincide with the predicted arrival times for a 5.4 EQ on the Southern East Pacific Rise.
After a lengthy initial alignment period, we are back in Observing mode. Some issues I ran into that I had not dealt with before include MICH_DARK not locking and SRC alignment issues. Kiwamu advised to skip the MICH_DARK locking and align MICH by hand to the best of my ability. Kiwamu also advised with PRC locking which basically involved gross adjustment of PRM. After a couple of rounds of PRMI tweaking, we proceeded to NLN with little issue. A few SDF diffs were accepted:
Observing @ 19:47 UTC.
Attached are 30 minutes of trends of the mass positions for the STS2-B ground seismometer used for SEI sensor correction. With the U mass x3 out of position spec and the IFO down for environment, I decided to really try centering. With Jeff's guidance, we figured out what we were looking at and now understand the process. Included in the attachment is a clip from the manual about the centering--hold the AutoZ for 5 0.5 seconds and after 30 seconds it is switched from 1 sec period back to 120 second period. Process may need repeating.
Well, bottom line, the U mass is still not centered and looks worse than just not being centered. I attempted this six times.
The U mass position is the lower left trace ...EPICS_CH20. The other two traces show that they are not moved very far during and return pretty close to where they started after the centering. The big glitch off the graph about mid way is when I toggled the FP switch to UVW and the 5th center attempt looks different as I manually switch the instrument to 1 sec.
The U mass position just jumps back and forth from +-15000 counts, 4.57V [3.0518e-04 V/ct.] What does this mean, at best...maybe the seismo is not well leveled. I pride myself in this and I think it is well leveled but the bubble could be poorly adjusted. Could there be a big offset in the position readout? Seem like it would always be the same sign if that were the case. Could the mass be stuck or maybe more likely dragging? Maybe if the machine was out of level, makes some sense. Could it be worse and just toast? Yes.
The second attachment is a view of the machine from Passcal. It shows the orientation of the UVWs wrt 'North/East.' For LIGO we've oriented these as North being the Yarm and East is the X arm. The UVW axes of the machine angle upward so that U V & W motion contribute to the Z and X determined motions but Y has only V and W mass movement contributions. This may tie in nicely with my post yesterday 24414 where it looks like the Y signal is good but the X and Z not so much.
I again recommend moving (carefully) the STS2-C instrument from the HAM5 area to the area near the BS where the tilt motion is minimal for the LVEA. And then we can look at the U axis leveling of the STS2-B unit.
Oh yes, of course while doing this, the HPI and ISI sensor correction were ramped down and then returned easily with the SDF. Oh how I love RCG 2.9.6.
Changed the diskless node mount points for /opt and /opt/rtcds to the x1fs0 computer, rebooted x1psl0. All models restarted and are running as well as before. I'm moving other test stand computers to use x1fs0 for /opt/rtcds.
All DTS computers that used x1boot:/opt should now be using x1fs0:/opt. The x1boot:/opt is no longer being exported.
O1 days 93-97
No restarts reported for these five days. No CDS maintenance on Tues 22nd.
All DAQ components have been running for 22.8 days and counting (except broadcaster which was restarted for detchar channel inclusion).
Twas the night before Christmas, when all through the lab The wind it was howling and tilting the slab; Seismic was ringing and microseism was too high To lock the thing up was just not going to fly; The staff were all tucked safe in their beds While visions of more data danced in their heads; The wind it was blowing at just a mere gale When the side of the building sounded like it was pelted with hail; I sprang from my chair to see what was the matter Away to the door to discover what was the clatter; As I opened the door, it was ripped from my hand with such speed, There was just enough time to duck flying tumbleweed; So back to the control room I quickly retreated And sunk into the chair, forced to admit I was totally defeated; This Owl shift was done before it was over With little to do but hope tomorrow would find LHO again in Observing clover; The doors are all locked, and the Guardian’s state is in Down All that is left is to head back into town; But the operator was heard to exclaim as he drove out off the site HAPPY CHRISTMAS TO ALL, AND TO ALL A GOOD NIGHT!
Winds are gusting into the high 30 to low 40s. Seismic and microseism are still high and showing an upward trend. Per conservation with the run coordinator earlier this evening, I have decided to abandon the rest of this shift knowing tomorrow will be a better day. To all the LIGO staff and families - Happy Holidays and a very bright 2016. Apologies to Clement C. Moore for taking such liberties with his 1823 poem "A Visit from St. Nicholas".
Jeff Kissel likes this!
Title: 12/24/2015, Owl Shift 08:00 – 16:00 (00:00 – 08:00) All times in UTC (PT) State of H1: 08:00 (00:00), The IFO is down due to environmental conditions. Wind, seismic, and microseism are all still elevated. Will attempt relocking if conditions improve, if not, per Vern, will cancel the rest of the Owl shift and leave it for the Day shift.
TITLE: 12/23 Eve Shift: 00:00-08:00UTC (16:00-00:00 PDT), all times posted in UTC"
STATE Of H1: Down due to environment
SUPPORT: Evan G
SHIFT SUMMARY: I managed to lock PRMI a few times in the begining of my shift, but ever since then the winds have gotten worse and the useism hasn't gone down. If I was able to get ALS locked, it has been impossible to keep it for longer than 30sec.
INCOMING OPERATOR: Jeff B.
ACTIVITY LOG: none
Today VerbalAlarms crashed due to a ValueError, but luckily we didn't need Verbs today because of the environment.
The problem came from code I added on December 9 that would stop the tempurature alarms from repeating too often. I it would look through a date string for the 9th-11th elements, which should be the minutes, and then see if it was five minutes since the last alarm. This was good and all until the date changed from a single digit to two digits. That was something I should have seen, my fault. The reason it did not show up until now is because it was under a few conditionals that have not been met since the 9th.
The fix was two part. First I made a regular expression to look through the date string and find the minutes. This worked well, but increased the loop time a tiny bit more. So I then went for the much simpler solution and made a global variable 'minute' from when I make the date string. Added only one line of code. This worked even better.
This configuration has been tested and I have reloaded Verbs with the new code.
Locking seems to be impossible, I can't keep ALS locked. I've tried many different blend configurations but there is always a ton of movement. It seems to be well aligned but then will just drift back to 0 power or it will immediately lose lock. I think the enviroment has got us for now.
useism ~1.0um/s. Wind <35mph.
Ryan, Jim, Dave
we built a new NFS server called x1fs0 today. It has a ZFS file system installed on two 1TB HDD. Ryan introduced us to the wonders of ZFS on linux. This machine is a new build Ubuntu 14:04LTS. We are currently rsyncing /opt from x1boot to x1fs0 (should complete this evening). Tomorrow we will test if we can move the NFS function off of x1boot over to x1fs0 in preparation for doing the same on H1 post-O1.
Ryan, Jim, Carlos, Dave:
Following the failure of the Tandberg tape backup, as a short term fix we built a 2TB file system on cdsbackup so we could perform a disk-to-disk-to-disk backup of /opt/rtcds and /ligo (replacing the current disk-to-disk-to-tape). With Ryan's help we built a 2TB ZFS file system on cdsbackup (a ZFS mirror on two 2TB disks). I am currently rsyncing /opt/rtcds from h1boot-backup to cdsbackup over the private backup network. This will take many hours to complete. I have verified (with Observium) that this traffic is going over the private backup network and not over the 10.101 or 10.20 VLANS, so it should not impact on the front ends or control room.
I have managed to lock PRMI two times, but could not advance from there. Since my two locks, it has been harder to keep ALS locked. Not going so well.
useism still ~1um/s, wind <30mph.
Today Ken (S&K Electric) and I replaced 5 heating elements in Zone 2A and 2 in Zone 1B. There were several elements burned out in Zone 3A so we relocated the remaining elements in order to bring these up sequentially as they were designed to do. These elements are a different style and we did not have replacements. I will order more after the first of the year. With the exception of 1A, which has some issues that will be addressed after the break, we are now able to turn on heat in all of the zones.
Manually filled CP3 at 22:43 utc, opened the fill valve by one full turn, time to see LN2 out of the exhaust 4:00 minutes.
We've continued to investigate activity around 74 Hz in order to explain the coherence line between H1 and L1 generated by stochmon.
Attached is a comprehensive set of results from the coherence tool (the full set of results can be found here). The coherence tool plots coherence with the local (H1 or L1) h(t) channel, averaged over a week with 1mHz resolution. The slides attached show the channels showing coherence structure around 74 Hz over the course of O1. You can find relevant plots via a download link in the attached slides.
Has any more digging gone in to investigating my suggested mechanism for coherence at this frequency (see LHO aLOG 24305)? Does this channel list support of refute the claim? Tagging the SEI, SUS, AOS, ISC, and SYS groups again.
There was also a 4.8 near Lakeview, OR at almost the same time.