model restarts logged for Mon 08/Sep/2014
2014_09_08 12:41 h1fw0

unexpected restart of h1fw0.

As the title says.  I replaced the ITMy oplev laser today and there is no change.  Data looks exactly the same as in Keita's previous alog.  Will continue to investigate tomorrow.

GV20 to have been closed next week anyway for X2 pressure accumulation 

Found h1nds1 had died about 15:49 PDT.  Rebooted computer, as it was unresponsive.  Note that the daqd process for h1nds1 had restarted itself at 15:30 PDT.

08:04 Karen - mopping at exit door in LVEA.

08:21 Aaron and Fil to LVEA to pull power cables for Illuminators and cameras from HAM3 F rack to Biergarden. Also, install junction box on top of HAM3 field rack. Drilling will be done and vacuums will be used.

08:25 Hugh to End stations to check on HEPI pumps.

08:47 Fire Dept on site to do annuals on fire extinguishers

09:02 D Barker - out to LVEA to start the Quad tst stand FE

09:59 Quad tst stand back up. Dave and Jim to restart OAF

10:11 Cris to End-X 

10:20 Richard out to LVEA by Fil

10:35 OAF is back up

10:47 J Kissell taking down HAM4,5 ISI to recompile model to include OpLev

10:57 Richard out of LVEA

10:58 P King into H1 PSL enclosure to execute work permit 

11:02 Hugh back from end stations

11:06 Jason out o LVEA to swap out ITMY oplev LASER

11:07 D Barker - DAQ restart

11:11 Praxxair called 10 minutes ahead of delivery - en route

11:55 Jason out for AHM 

12:00 All HANDS MEETING

13:43 Jason back into LVEA -replace bad power supply at IMTY oplev

14:27 Alistair into LVEA to "play with LASERS" - his words.

14:50 Mitch into the LVEA

15:00 Mitch out of the LVEA

15:45 Jason out of LVEA - apparently successful

BrianL Hugo JimW Hugh

When I collected HAM4 TFs 3 Sept, a deep zero here seen in the first plot as compared to the second, caught our eyes and it looks to be a problem possibly in the HEPI Isolation loop.

Here is the transverse controller from HAM3 4 & 5 in the attached.  The HAM3 controller is the top graph, appears to be done by hand and at a UGF of 2hz.  The middle graph is HAM4 and are the HAM HEPI Generic Loops developed by Hugo w/a ugf of 5hz and implemented by Hugh.  The bottom graph is HAM5 and again like HAM4, are generic and installed by me.  It is surprising that the HAM4 looks more of an issue than the very sharped peaked HAM5; but, the shoulders of the peak are higher at the ISI zero for HAM4 and the zero at HAM5 is below 6hz rather than above.  That said, the HAM5 HEPI controller is obviously equally problematic given this gain peaking.

Bottom lines--Generic loops while easy to install, should be carefully checked

Is a 5 hz loop best, warrented, needed?

Post HEPI turn-on ISI TFs should be collected and thoroughly checked.

Apologies to my above co authors--they have not been consulted for this entry.

J. Kissel, D. Barker

Completing the work I'd started with the HAM4 & 5 table optical levers (see LHO aLOG 13785), I recompiled, reinstalled, restarted and restored (4R'd) the HAM4 and 5 user models (h1isiham4, h1isiham5) to install the table oplev infrastructure. 

Note to the seismic team: we should modify the generic HAM-ISI overview screen to include a link to their respective oplev screen, instead of having these be separate line items in the sitemap pull down menu. Even better -- if anyone ever intends to *use* these levers -- we should make the infrastructure like the rest of the QPD / Optical lever world, and use the QPD Simulink library part, as well as the same screen. 

*ahem*...

Detailed list of activities
- Ramping down each ISI to the "READY" state, turning off the master switch
- Capturing  a new safe.snap for each (and committing the result to the userapps svn repo)
- Recompiling, reinstalling, restarting, and restoring the HAM4 and HAM5 models
- Restarting the DAQ / frame builder / data concentrator -- this failed, because of a duplicate channel naming bug in the HAM4 model 
- Fixing the bug in the HAM4 model
- Recompiling, reinstalling, restarting, and restoring the HAM4 model
- Restarting the DAQ / frame builder / data concentrator
- Turning on the HAM4 & HAM5 master switch, and returning HAM4 to "HIGH ISOLATED" and HAM5 to "ROBUST ISOLATED" via guardian.
- Committing the updated models to the userapps svn repo
- Adding the HAM4 and HAM% optical lever screens to the corner station ISI pulldown menu on the sitemap
- Committing the new sitemap to the userapps svn repo.

Yesterday people found that PR3 and BS seemed to move when PRC is carrier locked.

For PR3, there's no feedback and we see a clear correlation between the carrier power (sorry the first plot doesn't have the carrier power, only the AS power) and both OL and witness OSEMs. All in all PR3 oplev seems to slowly go negative in PIT after the carrier lock and goes back positive after lock loss, though this doesn't completely describe the behavior shown on the first plot.  PR3 DC OPLEV servo should be able to counter this if needed.

For BS it's somewhat more difficult to see partly because of the length feedback, but there's definitely a correlation between the carrier buildup and the BS oplev.  Unlike PR3, every time there's a carrier lock OPLEV  PIT goes positive and slowly comes back. BS oplev doesn't seem to be well correlated to the mean length feedback, so this is not the effect of the DC length push tilting the BS.

An RCG problem with 2.8.4 and later for models using the ramped mux matrix resulted in some of h1lsc's slow channels being channel shifted in the DAQ. This only affects 16Hz slow channels, fast data is stored correctly.

The build release pointer were changed from 2.8.5 to 2.8.3, the h1lsc model was recompiled and installed, the pointers were then reset back to 2.8.5.

h1lsc model was restarted. Due to a change with its INI file, the DAQ was also restarted.

We verified the slow data is  being stored correctly.

WP4832 Cyrus, Jim, Arnaud, Dave

We restarted the h1susquadtst front end in the LVEA. We took the opportunity to remove the RFM 5565 card from the CPU (a remnant of the H2 one-arm-test). The two models running on this front end (h1iopsusquadtst and h1susquadtst) were added to the DAQ and the DAQ was restarted.

On Monday I took Arnaud's h1susquadtst.mdl model and removed all IPC parts. Some inputs on the lower stages were reconfigured to match an ITMY suspension.

Yesterday, somebody started implementing DC-coupled PR3 OL servo actuating on M2 stage, so I made it usable.

Right now it's only for PIT (H1:SUS-PR3_M2_OLDAMP_P), and the filter is a simple 1mHz pole with 0dB DC gain, a 0.6Hz-ish elliptic low pass, and a 74dB gain (this gain is the inverse of the measured DC gain from the M2 OLdamp filter output to the OL PIT,  about +2E-4 rad/count).

With all filters on, the filter gain is equal to the DC gain, and the UGF is equal to 1mHz multiplied by the filter gain.

For the moment the filter gain is set to -10, so the UGF is 10mHz.

To turn it off, turn the filter output off, not the input.

To change the set point, change the filter offset.

WP4834 Jim and Dave

We replaced the 16bit DAC card in h1oaf0 (PCIX card on PCIe adapter) with a PCIe DAC card. This is the second DAC card in h1oaf0, the first is an 18bit DAC. Procedure was:

• stop running models
• take unit out of Dolphin fabric (local command)
• power down CPU
• power down IO Chassis
• replace card
• power up IO Chassis, power up CPU

This does not close out the WP, we have SUS EY and SEI EX still to complete.

The ODC model seems to be down, which took down the mode cleaner guardian since it is writing to some ODC channels.  I commented these out with Jamie's approval, since we don't want the ODC to take down the IFO if its not running.

Trying to get this in before the SEI T&C call. Pictures are from open loop measurements of ITMX. Foton pics are comparing T240 blends and X loops from ITMY and ITMY.

Lisa, Sheila, Alexa, Nic, Jamie, Kiwamu

We managed to lock the PRMI on the sidebands tonight. We estimated the servo gain settings from the gains in the carrier-locked condition. The lock was stable and lasted for more than 40 min.

Also, we saw an angular drift in pitch of PR3 when the carrier was locked. This behavior was consistently observed.

(Carrier lock)

We intensively worked on the carrier-locked PRMI in the daytime and we could repeatedly lock the PRMI on the carrier. The script which Alexa wrote is now interpreted in a guardian code by Sheila. It uses the variable finesse technique. The code is available but it is still a working progress. 

At about 3 pm local, we noticed that the motion in the Michelson was too large for us to keep locking the PRMI. At that time, Fil and Aaron were working around the beer garden, but this may not be a direct cause of the Michelson fluctuation because we did not see a significant improvement after they left. At this particular point, we could not keep closing the Michelson locking loop because of too large angular motion in the Michelson. This large motion gradually disappeared some time later and we could get back to the locking activity in the evening.

(PR3 tends to drfit in pitch)

When we locked the carrier-PRMI, we noticed that the power build-up degraded on a time scale of approximately 10 minutes or so. After some investigation, we identified it to be PR3 which drifted only in pitch by about 0.5 urad. We repeated the same observation a couple of times. PR3 always tends to drift to the negative side in the oplev counts. After a unlock event, it tends to go back to the original angle on a similar time scale. This indicates some kind of thermal issue. In fact, I did not observe a similar effect when the PRMI was locked on the sidebands. Sheila implemented an oplev servo on PR3 to mitigate the issue. In parallel, we should start working on some ASC loops.

Also, the BS oplev showed an approximately 0.5 urad drift once, but this did not seem to correlate with the power degradation. Perhaps this was simply some readout noise or something.

(Carrier lock without the variable finesse technique is difficult)

I still don't understand why this is so difficult, but locking the carrier-PRMI without the variable finesse turned out to be difficult. Since we already knew the right gain settings based on the variable finesse condition, we tried locking the PRMI directly. However, I succeeded in locking it only once. Plus, the lock did not last more than 20 seconds or so. Probably this was because I did not have the power normalizations which make the servo loops less sensitive to the angular fluctuation in the PRMI. This needs further investigations.

According to the variable finesse technique, we estimated the right MICH gain with ASAIR_RF45_Q to be -7, and the PRCL gain to be -0.2 with REFL_A_RF45_I. When I obtained the short lock, the gain settings were -9 and -1 for MICH and PRCL respectively. I used a trigger and triggered filters for both loops. It was around 4:23:56 in UTC.

(Sideband lock)

We then moved onto locking of the sideband-resonant PRMI. I simply flipped the sign of the PRCL loop to do this. But, this did not work -- I did not get a short lock. So I suspected some kind of cross-coupling in the sensing matrix and started trying to the other REFL sensor, REFL_A_RF9, which should be less sensitive to the Michelson motion. I measured the difference in the readout gain of the RF9_I and RF45_I by locking the carrier. The RF9 was smaller by 13 dB than the RF45. Taking this into account I tried some gain settings in MICH and PRCL while keep using ASAIR_A_RF45_Q for the MICH control. I attach a trend of some signals below to show the PRMI had been locked for a while, more than 40 min. Note that I did not have to re-align an optic to maintain the lock. I did not see drift in PR3 in this configuration.

The first acquisition was made with a MICH gain of -10 and PRCL gain of 0.7. These numbers are not far from the expected. Once it started locking, I tuned up the alignment, demodulation phases, UGFs and locking thresholds. I attach a screenshot of the whole settings, MICH  and PRCL open loop transfer functions.

It seems that lock acquisition is smoother with a slightly lower gain in MICH. For example, I kept using a MICH gain of about -7 for locking and increased it to the nominal gain of -17 once it was locked.

J. Kissel, S. Sachdev, K. Venkateswara

Krishna discovered that the Beam Rotation Sensor software had stopped outputting data around Sep 4, Thursday, night ~11:30 PM (~1093890000). Unclear why the software had failed, but it meant we had take a trip to the X-end to restart the software.

Instructions to restart the software:
(1) Head into computer users room, and wake the windows laptop sitting on the shelf ~1m up in the last rack on the left.
(2) Note the "rate" number in the upper left corner -- this is actually the DC value to which the BRS has drifted.
(3) Close the running analysis screen, by exiting out of the window (the red x in the upper right corner). This stops the software.
(4) Find (ctrl+F) the line in the code with the comment "DC subtract here," and change the value that's subtracted from "refLP[2]" to the "rate." Subtracting off the DC component of the signal helps reduce the impulse sent to the 1 [mHz] high-pass filter, therefore reducing the ring-down time upon start-up.
(5) Save the updated code (ctrl+S).
(6) Restart the software by hitting the "|> Start" button in the top-middle.
(7) Check that you now see reasonable rotation sensor signals in the raw ADC channel, H1:ISI_GND_BRS_ETMX_RY_INMON.

The DC value we subtracted was 1227 [ct]. The second attachment is pictures of the screen for steps 4&5 then 2&3.

In addition, I was showing Surabhi around the PCal components, so we rung up the BRS. So we spent ~30 minutes damping the suspension with our mass. This was only my second shot at this flying solo, so it took a little while to remember how massive I am. Instructions supplementing the notes from LHO aLOG 13538,
To damp:
- Open up a StripTool watching H1:ISI_GND_BRS_ETMX_RY_INMON close enough to the BRS that you can see it, but still an appreciable distance away (I plug the workstation into the wall socket on the West side of the BSC9, and push it as far North as I can).
- When the tilt signal is just after the maximum on the upper-half of its sign wave, stand close to the North (+X) side.
- The tilt signal is influenced artificially by the 1 [mHz] high-pass filter, which causes the signal to turn up deceivingly fast as one stands near the BRS -- don't wait until the trough to step away from the BRS.  
- As the resonance gets damped, you need to stand near for less and less time
- +/- 500 [ct] amplitude = need to stand on one side for a full half cycle (~30-50 seconds), +/-50 [ct] amplitude = only a few seconds at a time
- +/- 25 counts is good enough. 
- Once the tilt signal is at +/- 50 [ct], I (at ~180 [lbs], or ~80 [kg]) need only stand as close as the Southwest HEPI pier for 5-10 seconds to create the right amount of damping force.

First attachment is a time series of our trial and error attempts at figuring out the above procedure.