h1sush56 18bit DAC upgrade
Dave, Jim
We swapped all 5 DAC cards in h1sush56 with the upgraded firmware versions. The DAC cards were moved within the IO Chassis left by 4 slots to distance them from the new DC power board (D1301004). The 4th DAC card failed its autocal. We replaced this card with a new one, and it also failed autocal. We swapped the 4th and 3rd cards to see if this is a card or slot issue, now the 3rd card failed autocal. We put another new card into the 3rd slot and this time the calibration worked, so two cards which fail autocal have been detected.
h1guardian0 reboot
Dave
I rebooted h1guardian0 because its load average had risen into the 30's. This time the load average started around 12 instead of 7 before, so Ganglia was immediately complaining about overloading. It has since risen into the 20's, so rebooting is now short term relief.
h1sush2b reboot
Jim, Dave
h1sush2b had frozen up around 6pm last night. It required a reboot, which we knew would glitch all Dolphin connected computers so it was delayed until today's restarts. When we rebooted it this morning, we then restarted all corner station SUS, SEI and ISC. This in turn performed the autocal of all 18bit DACs which we indended to do today.
PEM ADC DQ channels
Dave
General ADC channels for occassional PEM work are being written to the frame. I set their gains to 0.0 to ensure they compress out in the frame when not in use. I changed SDF accordingly.
h1lsc new filters
The pending filter module change for h1lsc was applied today after the Dolphin glitch. The filter module H1LSC_SRCLFF has a rolloffs fllter, which moved from the 5th to 10th location within the filter module.
GRB/SN notification code
Dave
This code has crashed when the offsite networking went down. I upgraded h1fescript0 and restarted the code.
Leap Second
Stefan, Greg, Patrick, Jim, Dave
we tested that the CDS systems handled the leap second addition correctly. All systems did so, but surprisingly conlog crashed (Patrick is investigating). Greg confirmed frames and correct. Dave and Jim check NDS clients dtt and dataviewer.
Clocks on control room wall did not show the 23:59:60 second when applying the leap second. Rather they skewed ahead on second over the following two minutes. The one showing local time did this first, so for about a minute the two clocks differed by one second until the UTC clock caught up.