Last week (alog80749) the LOCKLOSS_SHUTTER_CHECK guardian node was stuck after not receiving the data it asked nds for. We've seen this happen in the past and I've written some wrapper functions to help with this - (userapps)/sys/h1/guardian/timeout_utils.py. While looking through the code with Sheila and Dave separately, it definitely seems like this node has been edited a bit and has been left in an odd state in need of cleanup. So I:

• Removed the unnecessary counters and other unnecessary code in the CHECK_SHUTTER state
• Added the timeout function calls to the nds data request and some logic around it
• Changed the edges a bit for slightly better state flow
• Added a few more comments, doc strings, and notifications for clarity

We also noticed that ISC_LOCK doesn't actually look directly at this node before powering up anymore. I looks like it was commented out in Dec 2018 for some SRY testing, then never put back in. The node was still watched by the top level IFO node, so it would still have stopped us from Observing, as it was the case last week. I moved that bit of code to the AS_SHUTTERS state where we already check the FAST_SHUTTER node (this node runs a test on the shutter before power up vs LOCKLOSS_SHUTTER_CHECK will check if it fired during the last high power lock loss). This way we won't power up if there was no GS13 kick on the last high power lock loss.

I tested the LOCKLOSS_SHUTTER_CHECK node by running it through the checks normally, and with the value of the GS13 kick incredibly low so it would flag a failure. This node is loaded and ready, and I reloaded ISC_LOCK with the new conditional in the AS_SHUTTERS state, though I have no yet ran this state since we have no IFO right now.