Reports until 10:05, Thursday 11 May 2023
H1 CDS
david.barker@LIGO.ORG - posted 10:05, Thursday 11 May 2023 - last comment - 11:50, Thursday 11 May 2023(69502)
CDS Maintenace Summary: Thursday 11th May 2023

Mini-Maintenance Morning.

WP11183 h1omc remove unused ADC channels

Daniel, Dave:

A new h1omc model was installed. The old model was reading some ADC channels and immediately terminating them. These have been removed. No DAQ restart was required.

WP11188 PSL FSS add NPRO TEMP fast channel to DAQ

Jason, Robert, Adrian, Dave:

The PSL FSS common model (pslfss.mdl) was changed to add the output of the NPRO_TEMP filter module to DAQ at the lowest rate of 256Hz. DAQ restart required, but since this is just an INI file change I did not need to restart the model, I did a DAQ LOAD instead.

++: fast channel H1:PSL-FSS_NPRO_TEMP_OUT_DQ added to the DAQ

WP11190 PEM add HAM7 FC1 ACC to h1pemcs and DAQ

Robert, Adrian, Fil, Dave:

A new h1pemcs model was installed. We have repurposed 3 channels from the 6th ADC to read the ACC_HAM7_FC_[X,Y,Z] channels. These were originally read by generic filter-modules and not acquired by the DAQ.

The X, Y, Z channels are being acquired by DAQ at 8kHz.

Latest SQZ models

Daniel, Dave:

Daniel's latest h1ioplsc0 and h1sqz models were installed. DAQ restart was required.

DAQ Restart

Dave:

The DAQ was restarted to support the above changes. FW0 was restarted at 08:55 and at 09:00 when it was attempting to write all three types of frame file its daqd crashed and restarted. It has been stable since.

GDS0 needed a second restart. 

No problems with the 1-leg restart.

Comments related to this report
david.barker@LIGO.ORG - 10:17, Thursday 11 May 2023 (69504)
Link

Thu11May2023
LOC TIME HOSTNAME     MODEL/REBOOT
08:50:14 h1oaf0       h1pemcs     <<< ACC HAM7 FC1


08:51:21 h1lsc0       h1ioplsc0   <<< New h1ioplsc0 and h1sqz
08:51:35 h1lsc0       h1lsc       
08:51:49 h1lsc0       h1lscaux    
08:52:03 h1lsc0       h1sqz       
08:52:17 h1lsc0       h1ascsqzfc  


08:53:33 h1omc0       h1omc       <<< remove unused ADC channels


08:55:03 h1daqdc0     [DAQ]
08:55:14 h1daqfw0     [DAQ]
08:55:14 h1daqtw0     [DAQ]
08:55:15 h1daqnds0    [DAQ]
08:55:22 h1daqgds0    [DAQ]
08:56:01 h1daqgds0    [DAQ] <<< 2nd GDS0 restart


08:59:50 h1daqfw0     [DAQ] <<< FW0 crash


09:02:31 h1daqdc1     [DAQ]
09:02:40 h1daqfw1     [DAQ]
09:02:40 h1daqtw1     [DAQ]
09:02:43 h1daqnds1    [DAQ]
09:02:51 h1daqgds1    [DAQ]
 

david.barker@LIGO.ORG - 10:45, Thursday 11 May 2023 (69508)
Link

DAQ Changes, num channels added/removed

model fast removed slow removed fast added slow added
h1pemcs 0 33 3 3
h1ioplsc0 0 1 0 11
h1sqz 0 1 0 11
h1omc 0 0 0 0
h1pslfss 0 0 1 0

 

david.barker@LIGO.ORG - 11:11, Thursday 11 May 2023 (69512)
Link

DAQ Channels Removed [name, bytes_per_sample, data_rate_Hz]

H1:PEM-CS_ADC_5_11_EXCMON 4 16
H1:PEM-CS_ADC_5_11_GAIN 4 16
H1:PEM-CS_ADC_5_11_INMON 4 16
H1:PEM-CS_ADC_5_11_LIMIT 4 16
H1:PEM-CS_ADC_5_11_OFFSET 4 16
H1:PEM-CS_ADC_5_11_OUT16 4 16
H1:PEM-CS_ADC_5_11_OUTPUT 4 16
H1:PEM-CS_ADC_5_11_SWMASK 4 16
H1:PEM-CS_ADC_5_11_SWREQ 4 16
H1:PEM-CS_ADC_5_11_SWSTAT 4 16
H1:PEM-CS_ADC_5_11_TRAMP 4 16
H1:PEM-CS_ADC_5_12_EXCMON 4 16
H1:PEM-CS_ADC_5_12_GAIN 4 16
H1:PEM-CS_ADC_5_12_INMON 4 16
H1:PEM-CS_ADC_5_12_LIMIT 4 16
H1:PEM-CS_ADC_5_12_OFFSET 4 16
H1:PEM-CS_ADC_5_12_OUT16 4 16
H1:PEM-CS_ADC_5_12_OUTPUT 4 16
H1:PEM-CS_ADC_5_12_SWMASK 4 16
H1:PEM-CS_ADC_5_12_SWREQ 4 16
H1:PEM-CS_ADC_5_12_SWSTAT 4 16
H1:PEM-CS_ADC_5_12_TRAMP 4 16
H1:PEM-CS_ADC_5_13_EXCMON 4 16
H1:PEM-CS_ADC_5_13_GAIN 4 16
H1:PEM-CS_ADC_5_13_INMON 4 16
H1:PEM-CS_ADC_5_13_LIMIT 4 16
H1:PEM-CS_ADC_5_13_OFFSET 4 16
H1:PEM-CS_ADC_5_13_OUT16 4 16
H1:PEM-CS_ADC_5_13_OUTPUT 4 16
H1:PEM-CS_ADC_5_13_SWMASK 4 16
H1:PEM-CS_ADC_5_13_SWREQ 4 16
H1:PEM-CS_ADC_5_13_SWSTAT 4 16
H1:PEM-CS_ADC_5_13_TRAMP 4 16
H1:SQZ-ADF_VCXO_OSC_DEMOD 4 16
H1:SQZ-FC_BEAT_RF_OSC_DEMOD 4 16

DAQ Channels Added [name, bytes_per_sample, data_rate_Hz]

H1:PEM-CS_ACC_HAM7_FC1_X_DQ 4 8192
H1:PEM-CS_ACC_HAM7_FC1_X_MON 4 16
H1:PEM-CS_ACC_HAM7_FC1_Y_DQ 4 8192
H1:PEM-CS_ACC_HAM7_FC1_Y_MON 4 16
H1:PEM-CS_ACC_HAM7_FC1_Z_DQ 4 8192
H1:PEM-CS_ACC_HAM7_FC1_Z_MON 4 16
H1:PSL-FSS_NPRO_TEMP_OUT_DQ 4 256
H1:SQZ-ADF_VCXO_OSC_DEMOD_EXCMON 4 16
H1:SQZ-ADF_VCXO_OSC_DEMOD_GAIN 4 16
H1:SQZ-ADF_VCXO_OSC_DEMOD_INMON 4 16
H1:SQZ-ADF_VCXO_OSC_DEMOD_LIMIT 4 16
H1:SQZ-ADF_VCXO_OSC_DEMOD_OFFSET 4 16
H1:SQZ-ADF_VCXO_OSC_DEMOD_OUT16 4 16
H1:SQZ-ADF_VCXO_OSC_DEMOD_OUTPUT 4 16
H1:SQZ-ADF_VCXO_OSC_DEMOD_SWMASK 4 16
H1:SQZ-ADF_VCXO_OSC_DEMOD_SWREQ 4 16
H1:SQZ-ADF_VCXO_OSC_DEMOD_SWSTAT 4 16
H1:SQZ-ADF_VCXO_OSC_DEMOD_TRAMP 4 16
H1:SQZ-FC_BEAT_RF_OSC_DEMOD_EXCMON 4 16
H1:SQZ-FC_BEAT_RF_OSC_DEMOD_GAIN 4 16
H1:SQZ-FC_BEAT_RF_OSC_DEMOD_INMON 4 16
H1:SQZ-FC_BEAT_RF_OSC_DEMOD_LIMIT 4 16
H1:SQZ-FC_BEAT_RF_OSC_DEMOD_OFFSET 4 16
H1:SQZ-FC_BEAT_RF_OSC_DEMOD_OUT16 4 16
H1:SQZ-FC_BEAT_RF_OSC_DEMOD_OUTPUT 4 16
H1:SQZ-FC_BEAT_RF_OSC_DEMOD_SWMASK 4 16
H1:SQZ-FC_BEAT_RF_OSC_DEMOD_SWREQ 4 16
H1:SQZ-FC_BEAT_RF_OSC_DEMOD_SWSTAT 4 16
H1:SQZ-FC_BEAT_RF_OSC_DEMOD_TRAMP 4 16
 

jonathan.hanks@LIGO.ORG - 11:50, Thursday 11 May 2023 (69515)
Link 

Adding some information regarding the FW0 restart.  The logs show it as missed data, skipping 18 cycles.

I'm adding some plots of timing and state in the system.  I find this interesting as usually with missed cycles we see abnormalities in the recv time values.  Long IO (which can also lead to backups) often show themselves with low buffer counts in the main buffer.  This is not the case.  Things look good right up until it dies that it all goes bad with no warning or buildup.
Images attached to this comment