Displaying report 1-1 of 1.
Reports until 13:41, Thursday 12 June 2014
H1 SEI
hugh.radkins@LIGO.ORG - posted 13:41, Thursday 12 June 2014 - last comment - 14:14, Thursday 12 June 2014(12337)
End X HEPI Pump Station restarted

The evidence I've been able to gather suggests the VFD (Variable Frequency Drive) saw an overvoltage while running at constant speed. OU3 is in fact an error code. This is something that has happened before, maybe a couple times a year. Don't really know what it means, will talk to EE. Since the troubleshooters (Rich & Jeff) shut down the servo despite it running before they pressed the reset button on the VFD, they were dead in the water until the servo was restarted (instructions in the OPS WIKI.)

In this situation with the Servo running and outputting max: 2048 coutns = 10 volts but producing no pressure, one could assume the pumps are not running due to a low fluid level trip or a fault at the VFD.  A trip to the Pump Station is warranted.

If the Red indicator on the DD Box is not illuminated, a fluid level trip or a power fault should be suspect.  In this case, reduce the output to the VFD by putting the servo in manual and stepping the output to zero or change the set point to zero.  Hit the green button on the DD Box, if it illuminates, ramp up the output of the servo manually or in auto mode by first reducing the set point to 5 psi and increasing it to 80 psi, do so slowly to minimize overshoot and risk of fluid level triping and over pressure valve opening.

If the Red Indicator on the DD Box is iluminated, it is likely the VFD is registering a fault.  Open the VFD box ( you must be EE qualified to do this!) and look at the VFD display, note the display.  Pressing reset from a VFD fault situation will restart everything so before doing so, as above, reduce the servo output to zero.  Once the servo output is zero, press reset and then as above, ramp up the output to 80psi.  Call me, I might be able to walk you through it remotely.

Comments related to this report
jeffrey.kissel@LIGO.ORG - 14:14, Thursday 12 June 2014 (12340)SYS
Notes from the conversation I had with Hugh and Rich this morning:

To restart controller (some of what's in the OPS Wiki, as Hugh suggests)

ssh controls@h1hpipumpctrl[ex,ey,0] (0 for corner station)
      (non-standard controls password -- check your secrets page)
cd target-ex/
sudo ./run

The rest is details of "After the controller is running and there are valid pressure signals, ramp the motor drive up to get the pressure near 80 psi and engage the servo. There will be some lag so slow down the ramping as you near 80 to be sure and not over shoot too far." Hugh mentioned that there're several ways to ramp up the servo, but here's how we did it this morning:
- Goto controller MEDM screen (resize if you see white -- it may not be dead)
- Flip "Servo Control" to Auto (button in middle, far right)
- In bottom left corner, select "Common Tweak Size" of 5
- Slowly (one click every 5-10 [s]) click "+" above "Set Point Tweak" to increase differential pressure set point to 80 [PSI] (a trace will begin to appear on the long black window once you get it within range)
- Once you get above ~50 ish [PSI] you can click a little faster, we're just looking to prevent large turn-on transients in the servo which'll move the platform. Hugh was explaining things to me in between clicks, so that's about the amount of time you need.


Rich also mentions that MIT has ways of automating this procedure such that "all you have to do is hit reset, then run" on the VFD control panel "and it just goes." I like this idea. This thing should also have a guardian wrapped around it, managed by the HEPI guardian, which in turn reports to the SEI chamber guardian.

Hugh also mentions there's an open Integration Issue (see II 242) in order to get LHO's pressure sensed at the chamber like LLO's. The last update says "within weeks" of 2013-Dec-19, but this has been delayed because of cable-pulling person-power. 

Also, there is some flaw internal to the HEPI Pump Servo board that means the "Pressure OK" and "Level Alarm" lights report mis-information. They should not be trusted.  Because this system is to be eventually converted to Beckhoff monitoring, attempts to make them LEDs functional have been abandoned. But so have attempts to convert to Beckhoff, due to similar person-power limitations.

We should get some person-power.
Displaying report 1-1 of 1.