Reports until 13:59, Sunday 16 November 2014
H1 CDS (ISC)
david.barker@LIGO.ORG - posted 13:59, Sunday 16 November 2014 - last comment - 13:20, Monday 17 November 2014(15092)
no reoccurrence yet of rapid setting of LSC PD_DOF_MTRX ramped mux matrix LOAD_MATRIX

Patrick, Dave

We have been scouring the houly conlog reports of EPICS PVs which change more than 500 times an hour for a reoccurrence of the rapid (many times a second) setting of the LSC ramped mux matrix which was observed last weekend. Unfortunately we have not seen anything yet, we will continue our vigilance.

FYI: here is the report from the last hour. (Remember that a 16Hz front end slow channel, if constanly changing, will give 16*3600 = 57,600 changes per hour.)

Process variables in conlog that have changed more than 500 time(s) in the last 3600 second(s):

H1:SUS-ETMY_L1_LOCK_L_RSET        57541        
H1:SUS-ETMX_L1_LOCK_L_RSET        57541        
H1:ALS-Y_CAM_ITM_SUM               2361        
H1:ALS-Y_CAM_ITM_YAW_POS           2361        
H1:ALS-Y_CAM_ITM_PIT_POS           2361        

Comments related to this report
rana.adhikari@LIGO.ORG - 00:13, Monday 17 November 2014 (15094)INJ, INS, SUS
Link

The L1 RSET frequency seems bad to me. I would interperet this to mean that the Guardian is resetting the filter history on that filter bank 16 times a second. Even if this filter bank is unused it indicates some rogue/bad Guardian script.

david.barker@LIGO.ORG - 13:20, Monday 17 November 2014 (15106)
Link

Rana is correct. Sheila, Alexa and Nic found the DOWN state of the ALS_DIFF guardian node was rapidly clearing the history of the ETM[X,Y]_L1_LOCK_L filters. This started at 8pm last Tuesday (11/11) and was ongoing until it was fixed a couple of hours ago. This highlighted a guardian logging issue with this node which Jamie is looking into.