Reports until 13:47, Friday 06 November 2015
H1 General
richard.oram@LIGO.ORG - posted 13:47, Friday 06 November 2015 (23160)
LLO Observing Time Loss due to HW/SW faults recorded in FRS for Month of Oct (10/1/2015- 11/3/2015)
LLO Observing Time Loss due to HW/SW faults recorded in FRS for Month of Oct (10/1/2015- 11/3/2015)

Total of 36.55 hours of OTL dues to faults during month of October 2015.

ID Product Comp Observation Time Lost (Hours) Status Resolution Summary Assignee Opened Changed
3928 LLO Detector Enginee general 2.50 NEW --- Alignment script issues, ITMY misalignment and REFL WFS trigservo loop not running adam.mullavey@LIGO.ORG Wed 06:39 Wed 12:37
3926 LLO Operators general 1.00 CLOSED PROCEDURAL Lock loss due to a CHARD excitation gary.traylor@LIGO.ORG Tue 19:28 13:39:07
3925 LLO Operators general 1.00 NEW --- Lock Loss due to aborting a measurement of (swept sine excitation) using DTT . gary.traylor@LIGO.ORG Tue 16:15 Tue 16:21
3922 LLO Detector Enginee general 0.25 WORKINPROGRESS --- Restarting the Guardian machine causes both ETM ISIs to trip. arnaud.pele@LIGO.ORG Tue 13:21 10:57:44
3916 LLO Detector Enginee general 0.10 WORKINPROGRESS --- Suspensions tripping arnaud.pele@LIGO.ORG Mon 15:39 Wed 09:28
3895 LLO Facilities general 2.50 CLOSED UNAVOIDABLE Power glitch - around 8:35PM local time timothy.nelson@LIGO.ORG 2015-10-28 02:35:34
3863 LLO Detector Enginee general 2.20 CLOSED UNRESOLVED Difficulties Locking Input Mode Cleaner (IMC) due to l1sush2a front-end DACKILL issue janeen.romie@LIGO.ORG 2015-10-23 Wed 15:30
3860 LLO Detector Enginee general 0.50 CLOSED DUPLICATE l1ecatc1 Beckhoff lost communication again janeen.romie@LIGO.ORG 2015-10-23 2015-10-24
3846 LLO Detector Enginee general 9.00 CLOSED UNRESOLVED TR CARM to TR REFL9 transition janeen.romie@LIGO.ORG 2015-10-22 2015-10-23
3830 LLO Detector Enginee general 0.10 CLOSED RESOLVED Tidal Servos janeen.romie@LIGO.ORG 2015-10-21 2015-10-26
3828 LLO Detector Enginee general 0.50 CLOSED UNRESOLVED ETMY ESD LVLN binary control mismatch janeen.romie@LIGO.ORG 2015-10-20 2015-10-22
3825 LLO Detector Enginee general 1.50 CLOSED PROCEDURAL ALS Laser head crystal frequency adam.mullavey@LIGO.ORG 2015-10-20 2015-10-23
3823 LLO Detector Enginee general 3.00 CLOSED RESOLVED Frontend laser tripped over the weekend matthew.heintze@LIGO.ORG 2015-10-20 2015-10-23
3808 LLO Detector Enginee general 3.00 CLOSED RESOLVED ALS-Y pitch oscillating adam.mullavey@LIGO.ORG 2015-10-17 2015-10-23
3796 LLO Detector Enginee general 0.25 PENDING --- Intent bit state change stuart.aston@LIGO.ORG 2015-10-15 2015-10-22
3793 LLO Detector Enginee general 2.00 CLOSED RESOLVED BS OpLev Sum below Guardian threshold stuart.aston@LIGO.ORG 2015-10-14 2015-10-16
3747 LLO Detector Enginee general 2.50 CLOSED RESOLVED Marginal CARM gain adam.mullavey@LIGO.ORG 2015-10-06 2015-10-16
3744 LLO Detector Enginee general 11.00 CLOSED RESOLVED ALS Y PZT-QPD servo instability adam.mullavey@LIGO.ORG 2015-10-05 2015-10-20

 

3928  is marked as new. ITMY did not misalign as requested by the script. PR2 and IM4 alignment REFL WFS not running. This is still being investigated.

3926 is marked as closed procedural. This fault was due to a human error. The actuation amplitude was too strong in the template and will be reduced to avoid this in the future.

3925 is marked as new.  The interferometer broke lock when aborting a measurement of the CHARD P open loop (swept sine excitation) using DTT. This is still being investigated.

3922 is marked as work in progress. To avoid Guardian nodes from hanging, it is now standard operating procedure to reboot the Guardian machine following a DAQ process restart, which is scheduled weekly at the end of the Tuesday maintenance period. However, there is a concern that there is a potential that this could re-occur should we need to restart these vulnerable Guardian nodes while the IFO is locked, resulting in a lock loss and more significant down-time. This is still being investigated.

3916  is marked as work in progress. Suspensions WD can trip during an alignment and be set to bad positions. Audible alarms should be set to avoid missing a suspension or SEI that tripped. These are not yet set up.

3895  is marked as closed unavoidable. Effect of glitch was limited i.e  PSL tripped but Front Ends did not. Recovery from power glitch was very smoothly done by LLO staff

3863 is marked as closed unresolved.  Alarms handlers have been set for operators to be informed should this issue arise again. We may wish to consider options for implementing something in Guardian.

3860 is marked as closed duplicate  of  Fault Report 3141 - MASTER: Beckhoff communication issue. We will monitor for a reoccurence

3846  is marked as closed unresolved. For some reason performing this transition only worked at a smaller CARM offset . It's not clear what necessitated this change. We will monitor for a repeat occurence.

3830  is marked as closed resolved. The settings were not saved correctly under the safe snap file which was restored during the model restarts -cf https://alog.ligo-la.caltech.edu/aLOG/index.php?callRep=21832-. The safe snap file has been corrected  to prevent a repeat occurence.

3828 is marked as closed unresolved.  Don't know what the root cause is yet. A note on a work around procedure has been added to the Operator Wiki.  We will monitor for a repeat occurrence

3825 is marked as closed procedural. This was an OOPS. Detector engineers miscommunicated interaction  between PSL work and affect on ALS.  Awareness has been shared which should prevent a reoccurrence

3823 is marked as closed resolved. This resulted in two action items one to look into widening  control limits on the front end watchdog which is programmed so that it trips at 15% loss of power from when it was activated and the other to close training gap on single operator on PSl restart procedure.

3808  is marked as closed resolved. Required spare PZT controller to be installed.   Root cause of the failure of the controller is still under investigation and being monitored for a  repeat occurrence.

3796  is marked as pending. This is not likely to be addressed until a future RCG update, and so it was recommended that we turn off monitoring of DACKILL channels in IOP models until the fix is available.

3793 is marked as closed resolved. The BS OpLev Guardian settings were widened which should prevent a reoccurrence

3747 is marked as closed resolved. The ISC_Lock Guardian has been modified which should prevent a reoccurrence

3744 is marked as closed resolved.  We will monitor for a reoccurence