Reports until 10:59, Monday 04 November 2024
H1 CDS
david.barker@LIGO.ORG - posted 10:59, Monday 04 November 2024 - last comment - 09:28, Tuesday 05 November 2024(81048)
Power cycle of h1psl0

WP12186

Richard, Fil, Erik, Dave:

We performed a complete power cycle of the h1psl0. Note, this is not on the Dolphin fabric so no fencing was needed. Procedure was

The system was power cycled at 10:11 PDT. When the iop model started, it reported a timing error. The duotone signal (ADC0_31) was a flat line signal of about 8000 counts with a noise of a few counts.

Erik thought the timing card had not powered up correctly, so we did a second round of power cycles at 10:30 and this time the duotone was correct.

NOTE: the second ADC failed its AUTOCAL on both restarts. This is the PSL FSS ADC.

If we continue to have FSS issues, the next step is to replace the h1pslfss model's ADC and 16bit DAC cards.

[   45.517590] h1ioppsl0: INFO - GSC_16AI64SSA : devNum 0 : Took 181 ms : ADC AUTOCAL PASS
[   45.705599] h1ioppsl0: ERROR - GSC_16AI64SSA : devNum 1 : Took 181 ms : ADC AUTOCAL FAIL
[   45.889643] h1ioppsl0: INFO - GSC_16AI64SSA : devNum 2 : Took 181 ms : ADC AUTOCAL PASS
[   46.076046] h1ioppsl0: INFO - GSC_16AI64SSA : devNum 3 : Took 181 ms : ADC AUTOCAL PASS

Comments related to this report
david.barker@LIGO.ORG - 13:00, Monday 04 November 2024 (81052)
Link

We decided to go ahead and replace h1pslfss model's ADC and DAC card. The ADC because of the continuous autocal fail, the DAC to replace an aging card which might be glitching.

11:30 Powered system down, replace second ADC and second DAC cards (see IO Chassis drawing attached).

When the system was powered up we had good news and bad news. The good news, ADC1 autocal passed after the previous card had been continually failing since at least Nov 2023. The bad news, we once again did not have a duotone signal in ADC0_31 channel. Again it was a DC signal, with amplitude 8115+/-5 counts.

11:50 Powered down for a 4th time today, replaced timing card and ADC0's interface card (see drawing)

12:15 powered the system back up, this time everything looks good. ADC1 AUTOCAL passed again. Duotone looks correct.

Note that the new timing card duotone crossing time is 7.1uS, and the old card had a crossing of 7.6uS

Images attached to this comment
david.barker@LIGO.ORG - 13:04, Monday 04 November 2024 (81053)
Link

Here is a summary of the four power cycles of h1psl0 we did today:

Restart ADC1 AUTOCAL Timing Card Duotone
10:11 FAIL BAD
10:30 FAIL GOOD
11:30 (new card) PASS BAD
12:15 (new card) PASS (new cards) GOOD

 

david.barker@LIGO.ORG - 15:18, Monday 04 November 2024 (81055)
Link

Card Serial Numbers

Card New (installed) Old (removed)
64AI64 ADC 211109-24 110203-18
16AO16 DAC 230803-05 (G22209) 100922-11
Timing Card S2101141 S2101091
ADC Interface S2101456 S1102563

 

david.barker@LIGO.ORG - 09:28, Tuesday 05 November 2024 (81067)
Link

Detailed timeline:

Mon04Nov2024
LOC TIME HOSTNAME     MODEL/REBOOT
10:20:14 h1psl0       ***REBOOT***
10:21:15 h1psl0       h1ioppsl0   
10:21:28 h1psl0       h1psliss    
10:21:41 h1psl0       h1pslfss    
10:21:54 h1psl0       h1pslpmc    
10:22:07 h1psl0       h1psldbb    
10:33:20 h1psl0       ***REBOOT***
10:34:21 h1psl0       h1ioppsl0   
10:34:34 h1psl0       h1psliss    
10:34:47 h1psl0       h1pslfss    
10:35:00 h1psl0       h1pslpmc    
10:35:13 h1psl0       h1psldbb    
11:43:20 h1psl0       ***REBOOT***
11:44:21 h1psl0       h1ioppsl0   
11:44:34 h1psl0       h1psliss    
11:44:47 h1psl0       h1pslfss    
11:45:00 h1psl0       h1pslpmc    
11:45:13 h1psl0       h1psldbb    
12:15:47 h1psl0       ***REBOOT***
12:16:48 h1psl0       h1ioppsl0   
12:17:01 h1psl0       h1psliss    
12:17:14 h1psl0       h1pslfss    
12:17:27 h1psl0       h1pslpmc    
12:17:40 h1psl0       h1psldbb