We had two rounds of restarts. The first was a full models+EDC+DAQ restart with a longer TW outage to copy rawminute trends to preserve recent lookbacks. The second was an unplanned restart to add an IPC between CRS and ISIHAM3 which needed a DAQ restart.
WP13365 CRS and ISI HAM2,3,4,5
New models for h1isiham[2,3,4,5] were installed and the DAQ was restarted. I was then found that an IPC between h1crsproc and h1isiham3 was both not running and was the incorrect type (a PCIE channel which should be SHMEM). Jim fixed the h1crsproc sender and h1isiham3 receiver. Because this channel was already in H1.ipc as a PCIE, I hand edited this file to remove it so the build of h1crscproc added it back as a SHMEM channel.
WP13366 PEM add 5th ADC channels to DAQ
New h1pem[ex,ey] models were installed which acquire the first 8 channels of the recently added 5th ADC to the DAQ at 2kHz. DAQ restart was required.
WP13346 Remove CP1 Overfill
The EDC master file was edited to remove the H1EPICS_CP1.ini file. EDC+DAQ restart was required.
WP13304 New HEPI Pump Control IOC
Patrick installed a new EY HEPI pump control system which has new EPICS channel names. A new H1EPICS_HEPIPUMPEY.ini file was installed. 120 new channels had old channel equivalents, the raw minute trends for this files were copied to the new names as part of the trend writer restarts.
DUST LAB2 removal from EDC
A restart Lab Dust IOC removed the LAB2 channels from its IOC. As a temporary solution I had added these channels to the edc_green_ioc dummy IOC running as a the service-host container. Today these channels were removed from both the container and H1EPICS_DUST.ini, and then removed from the EDC as part of the DAQ restart.
DAQ Restarts
The first DAQ restart included a rawminute trend file copy on both trend writers:
1 Restart models
2 Stop TW0 and TW1
3 Run copy script on TW0 and TW1, some new hpipump channels will be starting from where the old channels left off
4 DAQ 0-leg plus EDC restart (also restarts TW0)
5 DAQ 1-leg restart (also restarts TW1)
No major issues with this restart.
The second DAQ restart was just a model restart, no EDC restart was required.
FW1 spontaneously restarted 53 minutes later.
Restart Log:
Wed01Jul2026
LOC TIME HOSTNAME MODEL/REBOOT
09:12:31 h1iscex h1pemex <<< add ADC channels
09:13:05 h1iscey h1pemey
09:16:07 h1seih23 h1isiham2 <<< New CRS components
09:16:33 h1seih23 h1isiham3
09:17:20 h1seih45 h1isiham4
09:17:48 h1seih45 h1isiham5
09:22:02 h1daqgds0 [DAQ] <<< first 0leg restart
09:22:06 h1susauxb13 h1edc[DAQ] <<< EDC restart
09:22:07 h1daqfw0 [DAQ]
09:22:07 h1daqtw0 [DAQ] <<< TW0 restart after file copy
09:22:08 h1daqnds0 [DAQ]
09:26:02 h1daqdc1 [DAQ] <<< first 1leg restart
09:26:11 h1daqfw1 [DAQ]
09:26:12 h1daqtw1 [DAQ] <<< TW1 restart after file copy
09:26:15 h1daqnds1 [DAQ]
09:26:21 h1daqgds1 [DAQ]
09:30:52 h1daqfw1 [DAQ] <<< spontaneous FW1 restart
10:05:40 h1seih23 h1crsproc <<< Fix CRS IPC
10:06:14 h1seih23 h1isiham3
10:06:52 h1daqgds0 [DAQ] <<< second 0leg restart
10:06:59 h1daqfw0 [DAQ]
10:07:00 h1daqnds0 [DAQ]
10:07:00 h1daqtw0 [DAQ]
10:10:54 h1daqdc1 [DAQ] <<< second 1leg restart
10:11:05 h1daqfw1 [DAQ]
10:11:06 h1daqtw1 [DAQ]
10:11:11 h1daqnds1 [DAQ]
10:11:15 h1daqgds1 [DAQ]
10:11:59 h1daqgds1 [DAQ] <<< GDS1 needed a restart
11:02:37 h1daqfw1 [DAQ] <<< spontaneous FW1 restart
I found a copy-paste error in h1pemey with its ADC_4 bus selector. This was fixed and the model restarted at 14:01. No DAQ restart was required.
The filtermodules feeding the new ADC_1_[00-07]_OUT_DQ channels were set to pass the signal through and the SDF safe.snap files were updated.