hugh.radkins@LIGO.ORG - posted 12:01, Wednesday 10 September 2014 (13861)
EndX & EndY HEPI Pump Controllers Restarted to confirm database alarm addition goodness
no issues, goodness is good.
no issues, goodness is good.
Despite having everything green and resets pressed on all the HEPI screens(watchdog, dackill, iopdackill) there is no output to the Actuators. There are DK red lights on the CDS Overview. We (Richard & I) can't figure out how to clear this. He went away and return with "Restart the Model."
model restarts logged for Tue 09/Sep/2014
2014_09_09 09:11 h1iopsusquadtst
2014_09_09 09:11 h1susquadtst
2014_09_09 09:16 h1iopsusquadtst
2014_09_09 09:54 h1iopsusquadtst
2014_09_09 09:54 h1susquadtst
2014_09_09 10:34 h1iopoaf0
2014_09_09 10:34 h1oaf
2014_09_09 10:34 h1odcmaster
2014_09_09 10:34 h1pemcs
2014_09_09 10:34 h1tcscs
2014_09_09 10:46 h1dc0
2014_09_09 10:48 h1broadcast0
2014_09_09 10:48 h1fw0
2014_09_09 10:48 h1fw1
2014_09_09 10:48 h1nds0
2014_09_09 10:48 h1nds1
2014_09_09 10:54 h1isiham4
2014_09_09 10:54 h1isiham5
2014_09_09 11:05 h1lsc
2014_09_09 11:06 h1lsc
2014_09_09 11:07 h1dc0
2014_09_09 11:12 h1broadcast0
2014_09_09 11:12 h1dc0
2014_09_09 11:12 h1fw0
2014_09_09 11:12 h1fw1
2014_09_09 11:12 h1nds0
2014_09_09 11:12 h1nds1
2014_09_09 11:21 h1isiham4
2014_09_09 11:27 h1broadcast0
2014_09_09 11:27 h1dc0
2014_09_09 11:27 h1fw0
2014_09_09 11:27 h1fw1
2014_09_09 11:27 h1nds0
2014_09_09 11:27 h1nds1
2014_09_09 14:25 h1omc
2014_09_09 15:32 h1nds1
2014_09_09 15:50 h1nds1
2014_09_09 17:04 h1iopoaf0
2014_09_09 17:04 h1oaf
2014_09_09 17:04 h1odcmaster
2014_09_09 17:04 h1pemcs
2014_09_09 17:04 h1tcscs
2014_09_09 17:22 h1iopoaf0
2014_09_09 17:22 h1oaf
2014_09_09 17:22 h1odcmaster
2014_09_09 17:22 h1pemcs
2014_09_09 17:22 h1tcscs
2014_09_09 17:34 h1iopoaf0
2014_09_09 17:34 h1oaf
2014_09_09 17:34 h1odcmaster
2014_09_09 17:34 h1pemcs
2014_09_09 17:34 h1tcscs
maintenance day. Startup of h1susquadtst. Swap of DAC in h1oaf0, subsequent replacement of original DAC later. New ISI Ham4,5 code. Reversion of h1lsc to RCG2.8.3. New code for h1omc. Associated DAQ restarts. Unexpected failure of h1nds1.
(Jeff, Jamie, Sheila, Alexa, Lisa)
Tonight we used ITMX ISI with the guardian paused. We had damping on stage 2, and on stage 1 X, RX and RY. For stage 1 we used the level 3 controllers that are loaded and a gain of 2.
Our goal for the night was to lock PRMI on the carrier. However, we would lose lock when we aligned PRM after locking MICH on mid fringe. We must be missing some setting. We had found the FM2, FM3, FM4 were turned off (probably for locking on PRMI sideband) on LSC_POPAIR_A_LF. So maybe we are missing something else, but we just don't know what. Also the counts on LSC_POPAIR_A_LF_OUT have decreased to 1.8 from 3. To handle this, we adjusted the normalization matrix and triggering values, but this did not seem to work. The MICH mid-fringe OLTF was consistent with the previous measurements. We also slowed down the ramp for aligning PRM, but this did not help.
Jamie, Lisa
Jamie improved the tool he adapted from SEI to be able to plot multiple channels and rescale the axis all at the same time. It is now ready for public consumption.
The script which does the magic lives here:/opt/rtcds/userapps/release/sys/common/scripts/lockloss.
usage: lockloss plot [-h] [-w WINDOW] [-c FILE] [-o FILE] time
plot specific time
positional arguments:
time GPS time to plot
optional arguments:
-h, --help show this help message and exit
-w WINDOW, --window WINDOW
plot time window in seconds (default: '[-240, 5]')
-c FILE, --chanfile FILE
file with list of channels to plot, 1 per line ('-'
for stdin)
-o FILE, --outfile FILE
save plot to file
This is an example which uses a list of channels stored in my PRMI config file with some useful signals. I would be happy to attach this file to this entry, but apparently I can't.
In any case, it is just a list of channels...
lisa.barsotti@opsws3:~$ lockloss -c ~/lockloss/PRMI plot -w '[-200, 5]' 'tconvert Sep 9 10:34 UTC'
This is the lock loss that Kiwamu reported here , where the sideband build up is dominated by a 40 mHz oscillation in ITMX YAW.
The power fluctuations are kind of huge (75%), so the fact that the PRMI unlocked is not surprising, as Kiwamu was saying.
This tools will then talk to the Guardian and automatically provide a selectable list of lock loss GPSs.
The message of this entry is that, as long as NDS2 cooperates, this tools drastically improves the way we have been doing lock loss analysis in the last decade (and any analysis which requires zooming multiple channels, really).
AS Jason, Keita and I disscussed this morning, I left the BS oplev damping off for several hours today, from 9/9/14 17:00 UTC to 9/9/14 23:00
Minute trend of the BS OL over 8.5 hours around this off time. Around 21:30, there is a state change and the pitch DoF becomes much quieter.
Sheila, Alexa, Lisa, Kiwamu
Today we went back to locking SRY as described in 13726, to try make sure we have some reasonable crossovers for the SRC actuation.
First we adjusted the phase for AS_45, to -71.9. We also had to flip the sign, so we are now locking with a gain of -400 in SRCL.
we are using a gain of 1 in SRM and SR2 ISC INF.
After talking to Ryan we moved the 27 Hz notch to the SRM M2 lock filter, and disengaged the 27Hz notch in SRM ISCINF. This keeps the DAC from saturating at the 27 Hz vertical mode, without causing a notch in the open loop gain.
We have a ugf of about 30 Hz, measurements in the SRM path and the SRM M2 path are also attached. It seems that we do not ring up the vertical mode in SR2 as easily as we ring it up in SRM.
Causes outer "ability to drive" boarder to go white on overview screen. H1:ISI-ITMY_ST1_WD_MON_STATE_INMON H1:ISI-ITMY_ST1_MASTER_SWITCHMON H1:ISI-ITMY_DACKILL_STATE <-- This one shows red on middle click. H1:IOP-SEI_B1_DACKILL_STATE
[This work was done last week, but I forgot to submit this log.]
I added NOMINAL state definitions for all SEI systems, including HPI, ISI stages, and HAM and BSC chamber managers. All SEI guardian nodes were restarted to affect these changes.
NOMINAL state for HAM managers is "ISOLATED".
NOMINAL state for BSC managers is "FULLY_ISOLATED" (HPI and both ISIs isolated).
NOMINAL state for HPI is "ROBUST_ISOLATED", for both HAMs and BSCs.
NOMINAL state for ISI stages is "HIGH_ISOLATED".
Other than HAM 5 and 6, which are still being commissioned, everything except BS is set to be in the NOMINAL state, and is in the NOMINAL state.
We are currently operating the BS ISI with only stage 1 isolated, and stage2 left damped, corresponding to "ISOLATED_DAMPED" in the BS chamber manager (MICH locking has a tendency to trip the ISI stage 2 if isolated). However, I decided to leave the NOMINAL settings to the default for ISI_BS_ST2 and SEI_BS, so that we're reminded to address this issue down the line. Here's a crop from the GUARD_OVERVIEW, showing that the SEI_BS and ISI_BS_ST2 are not in their nominal states, as indicated by the far left OK indicators being orange:
We still need to add numeric indices for all the SEI states. This will require a bit more work for SEI since the HPI and ISI states are generated based on the particular configuration.
J. Kissel, J. Rollins, S. Dwyer, A. Staley While Sheila and Alexa began to lock PRMI on carrier, the HAM2 and HAM3 HEPI and ISIs tripped for an unknown reason. We'll leave this to people offline to figure out what happened. See attached actuator trip plots from the ISIs.
Again with the tripping. Only HEPIs this time. Only in the vertical direction.
Now can't bring up HAM2 or HAM3 HEPIs with out tripping. Took a look at HEPI pump controller -- the screen's not very non-expert friendly, but there's a red light at the pressure indicator ...
Pump System is fine--80psi at the output.
Sorry about the medm--been waiting for the long promised Beckoff system to upgrade channels etc. The Red light is the reservoir level, not pressure; the level switch is not hooked up to system.
This issue was (sadly) resolved with a restart of front-end processes; see LHO aLOG 13858. If DetChar's bored they can help CDS trace the problem by grabbing the exact time of failure. SEI Team -- is the CDS state word used in computing the "you have the ability to drive" outer ring of green?
Jeff--The first step of the out ring of the HEPI medm not being green is to go orange or something like that. This means only that the HEPI L4C have seen some saturations and the counter is no longer zero. The system is still operating normally even though the medm perimeter is not green.
I don't think this is used to calculate the ODC state bit--I'll investigate.--Jeff, I'm not sure what the CDS state words is. The ODC state word is green now on ITMX HEPI where the outer perimeter is orange and the Isolation loops are closed.
model restarts logged for Mon 08/Sep/2014
2014_09_08 12:41 h1fw0
unexpected restart of h1fw0.
As the title says. I replaced the ITMy oplev laser today and there is no change. Data looks exactly the same as in Keita's previous alog. Will continue to investigate tomorrow.
Yesterday people found that PR3 and BS seemed to move when PRC is carrier locked.
For PR3, there's no feedback and we see a clear correlation between the carrier power (sorry the first plot doesn't have the carrier power, only the AS power) and both OL and witness OSEMs. All in all PR3 oplev seems to slowly go negative in PIT after the carrier lock and goes back positive after lock loss, though this doesn't completely describe the behavior shown on the first plot. PR3 DC OPLEV servo should be able to counter this if needed.
For BS it's somewhat more difficult to see partly because of the length feedback, but there's definitely a correlation between the carrier buildup and the BS oplev. Unlike PR3, every time there's a carrier lock OPLEV PIT goes positive and slowly comes back. BS oplev doesn't seem to be well correlated to the mean length feedback, so this is not the effect of the DC length push tilting the BS.
Note that the plots posted above show that BS and PR3 moved about 1 to 1.5urad over 10 minutes.
Our PSL power is 10W, recycling gain at the moment is about 10.
These numbers sound as if this is a factor of 10-ish smaller than what the LLO people observed, but I'd appreciate any input.
These are the LLO numbers we were using for comparison: BS drift (without BS baffle): 5 urad @ 60 W on BS (LLO entry 9920); PR3 drift (without PR3 baffle): 12 urad / min (most likely still with 60 W on BS, entry 8960).
WP4834 Jim and Dave
We replaced the 16bit DAC card in h1oaf0 (PCIX card on PCIe adapter) with a PCIe DAC card. This is the second DAC card in h1oaf0, the first is an 18bit DAC. Procedure was:
This does not close out the WP, we have SUS EY and SEI EX still to complete.
Alastair, Jim, Aaron, Filiburto, Dave
Alastair later found that the new DAC card was not functioning correctly. We found that channel 12 was outputting 24V instead of 8.4V. We tried swapping out ribbon cable and interface cards, but eventually put the original DAC-on-carrier-card back in to get TCS back on track for this evening. We will investigate further later in the week.
Trying to get this in before the SEI T&C call. Pictures are from open loop measurements of ITMX. Foton pics are comparing T240 blends and X loops from ITMY and ITMY.
It looks like the blend filters installed at ITMX may be different. Using the plot_current_blends script, I looked at ITMY and ITMX. I need some help interpreting, but the complementarity plots don't all look the same.
HEPI & ISI are now isolating normally under Guardian--thanks Jim.