Reports until 22:26, Saturday 10 December 2016
H1 GRD
corey.gray@LIGO.ORG - posted 22:26, Saturday 10 December 2016 - last comment - 07:02, Sunday 11 December 2016(32435)
Continuing Investigation Into "sysecaty1plc2" SDF Issue With Knocking Us Out Of OBSERVING

Cheryl observed an instance of H1 being dropped out of OBSERVING due to SDF changes tracked down to the Computer/SDF Node:  sysecaty1plc2

(This was for the Yarm.  We noticed this same issue a week ago for the analogous Computer/Node for the Xarm:  sysecatx1plc2.)

I continued the work of figuring out who is our pesky channel dropping us out of OBSERVING.  The first thing I did was look at the (3) channels Keita found for X last week and see if the Y-arm counterparts changed today---found nothing in dataviewer.  I then ran the scripts Cheryl ran and came with the same result of seeing a change with the channel H1:FEC-1031_SDF_DIFF_CNT.  But this is just a name for a channel SDF uses.  

I then just went to where sysecaty1plc2 is on medm.  This is related to Beckhoff, so maybe the channel can be tracked down by snooping around medm land.  To get to a baseline/starting point, I went to:

SITE MAP / SYS / EtherCAT overview / H1 Y1 PLC2 /

From here you have several different subsystems (Als, Asc, Isc, Lsc, Sys).  So, I went through all of these subsystems and the screens nested within them.  The first thing I did was to find the "*_ERROR_FLAG" status box for each subsystem (it's green for all, and I reckon if there was a change to the system, it would go red).  So I grabbed this channel for all the subsystems mentioned above, and the only one which changed when we dropped from OBSERVING was the Als one.  I then played the same game--go into the nested windows within and trend "*_ERROR_FLAG" channels for each component within Als.  Ultimately, I ended up finding a single channel which had activity around the time in question.  It was found here:

SITE MAP / SYS / EtherCAT overview / H1 Y1 PLC2 / Als / Y / Fibr / Lock / Temeraturecontrols  (i.e. H1ALS_Y1PLC2_Y_FIBR_LOCK_TEMPERATURECONTROLS.adl)

And on this medm, the channel in question is:  H1:ALS-Y_FIBR_LOCK_TEMPERATURECONTROLS_ON

I'm not saying this is the ONLY channel which could be culprit for the OBSERVING drop, but this is one I saw drop out at that time (see attachment#1), BUT there is a caveat.  If I look at 20min before the Drop, the ALS channel in question had some similar drop outs (see attachement#2).  For the earlier one, the drops only lasted about 10sec (attachment#3).  For the drops which took us out of OBSERVING (attachment#1), after 15sec of drops, then we dropped out of OBSERVING (& overall the ALS ON switch went off/on for about 40sec).  So maybe the SDF changes have to happen for a certain amount of time before latching us out of OBSERVING?

As another check, I looked at the last 12hrs of this lock and the only time H1:ALS-Y_FIBR_LOCK_TEMPERATURECONTROLS_ON had these fits of turning OFF for a handful of seconds were in that 20min time period when we dropped out. 

Question:  Is this enough to warrant NOT MONITORING H1:ALS-Y_FIBR_LOCK_TEMPERATURECONTROLS_ON?  Or should we keep searching?

Images attached to this report
Comments related to this report
keita.kawabe@LIGO.ORG - 07:02, Sunday 11 December 2016 (32437)
This is enough to unmonitor that xhannel. To find out all activities that might be relevant, read the very last section of Cheryl's alog and run the lockloss script for ey ethercar plc2.