Reports until 15:58, Friday 05 April 2019
H1 CDS (GRD)
david.barker@LIGO.ORG - posted 15:58, Friday 05 April 2019 (48270)
stranded guardian git lock file caused ISC_LOCK problems

Sheila, Jenne, Dave:

After the ISC_LOCK guardian node was reloaded, it reported a "HEAD.lock file exists" error. The file in question was /ligo/cds/lho/h1/guardian/archive/ISC_LOCK/.git/logs/HEAD.lock which was last modified 12:11 Tue 02apr2018, this is the time we rebooted h1guardian1.

This file had zero permissions,

---------- 1 1010 controls       0 Apr  2 12:11 HEAD.lock

I deleted it as root on the NFS server. I also proactively scanned all the .git directories to see if any other lock files a lurking, there are none.