Displaying report 1-1 of 1.
Reports until 22:04, Sunday 30 April 2017
H1 DCS (DCS)
gregory.mendell@LIGO.ORG - posted 22:04, Sunday 30 April 2017 - last comment - 22:11, Sunday 30 April 2017(35924)
Problem with DMT H1 lock segments and redundant hoft at LHO
FYI:

1. Note that the good news is I see H1 low latency hoft making it to CIT, and it is being aggregated. Also it seems the low-latency analysis can determine the lock segments from the hoft frames. 

Thus, the problems listed below are something we can investigate Monday morning. An email has been sent to DASWG and the relevant persons about this.

2. However, the redundant copy of hoft written by the DMT seems to have
stopped today at:

Apr 30 11:21 PDT.

Much earlier today, the DMT logs report Input/output error like this:

$ tail  logs/H1_rename_Frames-2017.04.25-09.43.log

‘/gds-h1/dmt/frames/hoft/H1/.temp/H-H1_DMT_C00-1177574268-4.gwf’ ->
‘/gds-h1/dmt/frames/hoft/H1//H-H1_DMT_C00-117757/H-H1_DMT_C00-1177574268-4.gwf’
mv: cannot move
‘/gds-h1/dmt/frames/hoft/H1/.temp/H-H1_DMT_C00-1177574268-4.gwf’ to
‘/gds-h1/dmt/frames/hoft/H1//H-H1_DMT_C00-117757/H-H1_DMT_C00-1177574268-4.gwf’:
Input/output error

3. Also, thus summary pages show dropout of the H1 lock segments

https://ldas-jobs.ligo-wa.caltech.edu/~detchar/summary/day/20170430/

and nagios reports errors like,

https://dashboard.ligo.org/cgi-bin/nagios3/status.cgi?host=all&servicestatustypes=28

"Known segments in frames that aren't in database"

Thus, low-latency data is making it to CIT, I think with the correct
locked segments, but these are not making into the segment database.

This is something we'll investigate Monday morning.

 

Comments related to this report
gregory.mendell@LIGO.ORG - 22:11, Sunday 30 April 2017 (35925)

This is not related to previous problems with the calibration code not reporting lock after a data dropout.

This recent frame,

bash-4.4$ ls -l /scratch/frames/cache/hoft/H1/H-H1_DMT_C00-117764/H-H1_DMT_C00-1177648628-4.gwf
-rw-r--r--   1 40829    40829     392952 Apr 30 21:37 /scratch/frames/cache/hoft/H1/H-H1_DMT_C00-117764/H-H1_DMT_C00-1177648628-4.gwf

bash-4.4$ tconvert -l 1177648628
Apr 30 2017 21:36:52 PDT

reports a H1:GDS-CALIB_STATE_VECTOR of

67010559 = 0b11111111100111111111111111

which says H1 is in analysis ready mode.

Displaying report 1-1 of 1.