Reports until 06:01, Friday 28 April 2017
H1 CDS (CDS, SUS)
corey.gray@LIGO.ORG - posted 06:01, Friday 28 April 2017 - last comment - 07:46, Friday 28 April 2017(35857)
12:44 Out Of Observing: EX SUS FrontEnds DOWN!!

(I'm almost cutting & pasting from what happened almost 24hrs ago!  Yesterday it was EX SEI, Today it is EX SUS!)

At 12:44utc (5:44amPDT):

(attached is a screenshot of all the WHITE screens we have for EX.)

Images attached to this report
Comments related to this report
corey.gray@LIGO.ORG - 07:46, Friday 28 April 2017 (35860)

Tally of activities taken to recover 

  • Richard (& I) had trouble logging into EX Frontends (and any other ones); this was due to the controls password being changed.
  • Dave called in.  
  • 13:42 Richard heading out to EX
  • 13:42 Since we'll probably be rebooting the EX World, taking any Guardian Nodes I can access to a SAFE state (i.e. SEI, HEPI, etc.)
    • Since I did not see an obvious "SAFE" state on the SEI/HEPI/ISI Guardian Nodes (or anything in the SEI wiki), I called Hugh and he suggested I take the EX SEI Guardian Node to OFFLINE.
  • 1351 h1susex booted by Richard 
  • 14:05 Richard back to corner station (Dave must still be working on getting us back & GREEN at EX).
  • 14:11 Dave restarting models
  • When we lost EX SUS, the SUS & TMS bias sliders shifted.  Used Time Machine to find out values from ~5:05amPDT, but also trending Oplev signals (for etmx) & M1_DAMP pit & yaw signals (for tmsx).  The sliders shifted during the reboot this morning (and yesterday morning).  
    • 14:35  Starting with no light on the ALSx camera---tweaking sliders to restore signals above has brought back a spot on the ALSx camera!
    • (NO DITHER scripts used this morning!)
    • OK, working on getting stuff to a decent state for Jim, so that hopefully all he'll have to do is an Initial Alignment.