model restarts logged for Fri 15/Jan/2016
No restarts reported. h1tw0 continues to be down waiting h/w repair.
Slushy snowy morning here on site with useism slightly trending up (above 90 percentile at 0.7um/s), and the EQ band actually has been trending up slowly over the last 12 hrs from ~0.05 up to 0.2um/s. Winds are under 10mph.
H1 has been in Observing as TJ left it and it has been locked for 16hrs (for most of the night the range was hovering just under 80Mpc. Now it has been around 75Mpc for the last 2 hrs. On DARM, I can clearly see elevated noise in the 20-30Hz band, a line around 42Hz & also noise in between. POP A on the PRMI striptool & tidal channels on their striptool show oscillations at ~0.4Hz.
I believe I see Robert pulling onto site (as I was chatting with Landry).
I've tried contacting LLO Control Room (TeamSpeak & phone), but perhaps someone has just stepped out of the CR (According to their Summary Page, they someone has been working on bringing L1 up). Will touch base with them when they are available to coordinate double coincidence for a suite of hardware injections. In the meantime, Robert will be taking H1 for PEM injections soon. Robert has said that he can "stand down at any time to allow hardware injection work". Robert will be heading to an End Station for his PEM work.
TITLE: 1/15 Eve Shift: 00:00-08:00UTC (16:00-00:00 PDT), all times posted in UTC"
STATE Of H1: Observing at 78Mpc for 7hrs
SHIFT SUMMARY: Had one small lockloss but it came right up.
INCOMING OPERATOR: none
ACTIVITY LOG:
In full lock (with the POP beam diverter open), I rephased POPAIR9 so that the appearance of PRCL is maximized in I, and I rephased POPAIR45 so that SRCL is maximized in I (same strategy as was done for POP).
The changes are as follows:
I don't think POPAIR is used for any LSC at any point during locking or initial alignment, so there should be no issue here.
Then I left the beam diverter open and redid the LSC sensing matrix measurement. Times as follows (2015-01-16 Z):
I then left the beam diverter open and collected quiet data from 02:43:30 to 03:32:30. Kiwamu and I had discussed the possibility of cross-correlating the POP and POPAIR PDs in order to look for correlated vertex LSC noise below the PD shot noises (à la the usual DCPD technique for DARM). In the case of POP45Q (attached), the cross-correlation seems to show correlated noise out to high frequencies, at a level of a few below the POPAIR45Q shot noise.
Done with calibration for the day, but it will resume again in the morning.
Sheila, Jenne, Evan
We had another instance of the integrated ALS tidal offsets causing the IMC-F offloading to bump up against the rail.
We weren't sure how to turn on the bleed feature of the ALS filters, so we instead doubled the digital limiter on the COMM control filters.
I've schedule another 2 runs of the long burst injections and a CBC BNS injection for tonight. These will start at 20:10 CT (6:10 UTC), there are 5 burst injections and 1 CBC injection, meaning that the last one wil be at 23:50 CT (9:50 UTC). Here is the updated schedule: 1136952617 2 1.0 burst_GPS_76.259_ 1136953817 2 1.0 burst_GPS_76.262_ 1136955017 2 1.0 burst_GPS_76.263_ 1136956217 2 1.0 burst_GPS_76.264_ 1136957417 2 1.0 burst_GPS_76.266_ 1136958617 1 1.0 coherentbns1_1135135335_ 1136959817 2 1.0 burst_GPS_76.259_ 1136961017 2 1.0 burst_GPS_76.262_ 1136962217 2 1.0 burst_GPS_76.263_ 1136963417 2 1.0 burst_GPS_76.264_ 1136964617 2 1.0 burst_GPS_76.266_ 1136965817 1 1.0 coherentbns1_1135135335_
Nominal Low Noise 01:10
While relocking, the OMC_LOCK Guardian reported a "Wrong mode?" notification just before DC_READOUT. The OMC Trans camera looked great, so I wasn't sure of the issue. I looked through the code and H1:OMC-DCPD_SUM_OUTPUT needs to be > 14 for it to move on. This channel was around 13.0 +/-0.5. Jenne thought that the PSL power was slightly low, and suggested that I use the rotation stage to bump it up a few tenths of a W. Evan also gave his nod of agreement and I changed the rotation stage from a request of 2.000W to 2.100W. Nothing. Then from 2.100W to 2.400W. Success!
While I was adjusting this, Jenne was inputing the PSL power into a small box on the LSC_OVERVIEW (shot attached). The Guardian will normally write to this value, but since we were changing it, it needed to be updated. So for future attempts at this, make sure to be changing the value in that box as the PSL power changes to keep everything happy.
Robert was running inj at the time, but I'm not sure if that was it or not. Still investigating.
Attaching the generic lockloss plot. Nothing evident here.
ISI BS St1 and St2 WD tripped during the lockloss, so I have also attached the ACT plot. Robert was inj on ISI BS at the time so I suspect it was this inj that caused the lockloss.
After notifying the operator I drove past the CS @ 23:35 and arrived at Y-Mid station @ 23:38. IFO was out of lock.
Started filling CP3 @ 23:39, opened the exhaust check valve bypass valve, then opened LLCV bypass valve 1 turn, started to notice flow @ 23:47. and closed the LLCV bypass valve @ 23:48.
5 minutes later I closed the exhaust check valve bypass valve.
Started driving back from Y-Mid @ 23:53, arrived at the CS @ 23:56.
TITLE: 1/15 DAY Shift: 16:00-00:00UTC (08:00-04:00PDT), all times posted in UTC
STATE of H1: Getting back to NLN for Robert
Incoming Operator: TJ
Support: Jenne (help with locking)
Quick Summary:
Robert had H1 for a good chunk of the day.
Shift Activities:
TITLE: 1/15 Eve Shift: 00:00-08:00UTC (16:00-00:00 PDT), all times posted in UTC"
STATE Of H1: Nominal Low Noise, calibration measurements ongoing
OUTGOING OPERATOR: Corey
QUICK SUMMARY: Wind <10mph, useism 0.5um/s, CW inj running.
Thanks to TJ for taking the data yesterday during a break in PEM calibration locks.
Attached are the usual plots of the H1 ETMX and ETMY long trends of charge accumulation. No major surprises. I believe Kissel intends to switch signs of bias sometime ~soon.
Jeff K, Jeff B. , Jenne, TJ, Jim, Dave:
Around 4pm PST TJ reported that OMC had tripped and the watchdog could not be untripped. Jeff K. recommended a model restart. Unfortunately due to a communication problem we first mistakenly restarted the OMC model on the LSC front end (sorry OMC). Then we restarted the correct SUS-OMC model on SUSH56. This did not fix it. We then restarted all the models on SUSH56 (including the IOP). This did not fix it. We then stopped all models and only started IOP and SUS-SRM to do further debugging. (in the mean time the SWWD on the IOP had tripped SEI for HAM5 and HAM6). After some debugging we found that the PERL script sus/common/scripts/wdreset_all.pl was throwing an error about not finding the PERL CA LIBRARY. Jim tracked this down to a missing CaTools.pm perl module in the userapps/release/guardian directory. Turns out this file was removed from the SVN repository way back on 2nd March 2015 and the LHO working directory was only updated this afternoon by Jenne and TJ. This all nicely ties in with the watchdog resets working last night but not this afternoon.
In the mean time we had manually reset the watchdogs for SUS-SRM/SR3/OMC and SEI HAM5,6 and set the SDF back to OBSERVE for SUSH56IOP, SUSSRM/SR3/OMC and OMC.
For now we have manually copied the CaTools.pm file into userapps/release/sus/common/scripts to get the watchdog reset script working again.
This raises an FRS:
A perl module which is used by the watchdog systems has been deprecated. The watchdog system should be changed to no longer use PERL and instead use PYTHON (or perhaps BASH for exceptionally simple scripts).
We experienced a seemingly identical occurrence of this issue at LLO last Wednesday (see LLO aLOG entry 24156). However, as well as the SUS/SEI Watchdog reset scripts our initial alignment script was also affected, since it has Perl dependencies. It is still unknown how the symbolic-link to CaTools.pm became broken at LLO, see #4180.
Stuart, it was broken because I updated the same the same folder when I was visiting LLO. I am at fault for both of these CaTools.pm links being broken at both sites, though I had no idea that simply updating the SVN could cause this.
Thanks for shedding light on this mystery! I would suspect that svn'ing up pushed the changes to deprecate the Perl module sooner than was intended.
J. Kissel, E. Goetz Having a few new breakthrough ideas on the UIM actuation system (see LHO aLOG 24914), we explored whether we are modeling the [m/N] L1/UIM force to L3/TST displacement transfer function incorrectly. This was done by driving the UIM out to 600 [Hz] and measuring the response in DARM. Not only did we find the expected-but-not-yet-modeled wire violin modes at ~330 [Hz], 420 [Hz], and some at ~500 [Hz], but we found several bending-mode resonances at 111 [Hz] and 166 [Hz]. Indeed, upon first glance, we think the 111 [Hz] resonance is the remaining missing frequency dependence that explains the turn-up seen at 100 [Hz] in all previous measurements of the UIM to TST transfer function. We'll process in more detail some time in the future, but check out the attached screen shots and be amazed at how not 1/f^6 the L1 to L3 transfer function is. -------- We'd started by exciting the L1 stage via awggui in a broad-band fashion such that we could catch all of the wire violin mode frequencies watching DARM. As Evan mentions, we had suspected that these wire resonances -- documented in T1300876 -- were the source of the deviation from 1/f^6, and they'd just not been included in the SUS dynamical model -- [[EDIT -- Brett has now included them in the model, and they are a non-negligible effect; see LHO aLOG 24915]]. This broadband TF is shown in the first attachment. Black is the with excitation ON, and red is ambient (to distinguish the ~505 [Hz] fiber violin modes from the ~495 [Hz] TOP to UIM wire violin modes, and the Beam Splitter violin modes & 331.7 [Hz] calibration line from the Sus. Point to TOP wire violin modes). Only the peaks of the wire violin mode resonances are visible above the DARM noise; driving them any higher breaks the IFO's lock. Just in case, we drove down to the same ~80 [Hz] region, and BINGO! We also found new, unexpected resonances at 166 [Hz] and even as low as 111 [Hz]! (see second and third attachments) Our best guess for the source of these resonances are imperfect actuators. Perhaps the bending of the L-bracket that mounts the OSEM coil to the reaction chain's UIM (see D060375, see page 1 for the total assembly, lock at pg 14 for the L-bracket). Though, if it were these L brackets, I'd suspect there would be 4 independent resonances... also it doesn't look like enough moving mass to have resonance frequencies as low as ~100 [Hz]... dunno, will think more. Finding something at 111 [Hz], we then took a careful swept sign measurement covering it, and indeed, it looks strikingly like another piece of the UIM puzzle. (see 4th attachment). We also grabbed a PCAL to DARM transfer function over this frequency vector, so we can turn the TF into an absolute calibration later. For future reference, the templates for the swpet sine TFs are here: /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/PostO1/H1/Measurements/FullIFOActuatorTFs/ 2016-01-12_L1toDARM_FullLock.xml 2016-01-12_PCALYtoDARM_FullLock.xml and I attach screenshots of the awggui sessions used to excite L1 and PCAL in a broad-band fashion (DARM_IN1 ASDs during the broad-band excitations were taken using the standard wall FOM for DARM).
Weird about the 111 Hz and 160 Hz modes. If it is a mechanical mode of the UIM OSEMs, it is probably more likely to be the magnet-flag assembly, which makes a nice long cantilever, and is attached to the main chain itself. See page page 16 of D060375. Additionally, if for some reason the set screw holding the flag assembly in place is loose, you met get lower frequency modes.
The L bracket is on the reaction chain, so if the mode was in that it have to couple through to the main chain via the magnetic field gradient inside the coil; it's possible but one more step removed.
You could try exciting these modes one OSEM at a time, to see if it is coming from one in particular. If we get lucky, maybe we'll find there is a simple fix, like tightening a set screw.
I also wonder if these features exist on other test mass suspensions.
The 111Hz feature is very likely from the first internal mode of the UIM blades which is not surprising sicne it will ebt there at some level due to cross-coupling. The frequency is very close - see for example https://alog.ligo-la.caltech.edu/aLOG/index.php?callRep=16740 where the frequencies were measured at LLO for their quads to be in the range ~111 to 112 Hz. As for the 166 Hz, I don't have a good idea. It is not the second resonance of these blades. Lab measurements of such a blade here at Caltech give the second mode at around 325 Hz. Funnily enough this is ~twice the observed feature, but I can't think why we would see something at half the frequency of a blade mode.
I was asked by Jeff to fit the UIM data so that we can include the peaks at 111 and 167 Hz in our calibration model. After some struggle, I ended up doing an emperical zpk model which gave me the following parameters:
=========================
gain = 2.270401e-09
f:pole0 = 1.113398e+02
Q:pole0 = 5.596904e+14
f:pole1 = 1.950899e+02
Q:pole2 = 4.743158e+00
f:zero0 = -1.133450e+02
Q:zero0 = 6.220362e+01
=========================
In addition to these fitted parameters, I had fixed zpks which are 6 poles at 1 Hz and 1 complex pole at 166.7 Hz with a Q of 200. The attached shows a comparison of the fitting and data. I have used fminsearch to minimize a weighted residual. I didn't have an energy to compute the uncertainties in the estimated parameters.
One thing I don't like with this fitting is that the fitted model falls faster than the nominal 1/f^6 slope above approximately 160 Hz due to the extra poles that I put in to make the fitting better at frequencies below.
The code and resultant figure can be found at:
While investigating an issue with x1seibsctim04 Jim Batch and I noticed that the power supply on the x1seiex IO chassis was dead. We replaced the power supply with a spare unit from the self. The new power supply appears to be getting 5v (some leds where lit), but it would not turn on (the fans would start to spin up and then stop).
Further investigation is needed. For now x1seiex has been disconnected from the dolphin network and turned off.
FRS 4218
https://services.ligo-la.caltech.edu/FRS/show_bug.cgi?id=4218
From the FRS: --- Comment #1 from richard.mccarthy@LIGO.ORG --- Turns out the an ADC interface board had a short dragging the power supply down. Replaced the board and returned the unit to service.
Terramon after-the-fact notified us that there was a 4.9 Costa Rica quake at 17:23 with motion of 0.38um/s.
h1tw0 continues to be down (also causing a red EDCU)...not sure whether I need to keep reporting this. This comes up when operators run through the shift check sheet.