New Frame Writer Added To DAQ WP5455
Keith, Dave, Jim
The new frame writer was connected to the QFS file system this morning. We are still in a testing phase, so the machines don't have their final names at the moment. The MSR-QFS file system was removed from h1fw1 and added to h1fw2. The configuration for h1fw2 is to write science, commissioning, second trend and minute trend frame files (as it was when testing on local disks). The configuration for h1fw1 was changed to stop writing commissioning frames, second and trend frames. It now only writes raw-minute data files on its local SSD-RAID.
The only problem found in the transition is that with the new daqdrc configuration file h1fw1 would not start running. Comparing with the LLO file, we moved the line
-
set raw_minute_trend_saving_period=5;
to near the top of the file (line 3) and the daqd process would then run.
h1odcmaster, putting the TCS IPC channel back in WP5474
Ryan, Duncan, Dave:
The TCP IPC channel was mistakenly removed from h1odcmaster during last Tuesday's upgraded. We put the channel back in today. The model and the DAQ were restarted.
DAQ reconfigured for new EDCU files, major problem with DAQ found and fixed
Keita, Daniel, Jamie, Jonathan, Jim, Dave:
Several EDCU files were modified this morning:
-
Beckhoff: all the Beckhoff ini files were changed to add new timing components
-
Guardian's latest EDCU file with the new nodes
-
DAQ ini file had the new fw2 channel's added
-
FEC ini file was brought up to date with sus-pi models
-
a new GDS ini file added to record the IR in the DAQ
These changes went in with the DAQ restart at 12:39 PDT. This afternoon Keita reported that trended data is showing large 16Hz spikes in the SUS channels he was looking at. We looked at some PEM channels as saw the same noise in the full frames, second frames and minute trends starting at 12:39. We switched our NDS from h1nds1 to h1nds0 which showed the same problems, so it was not caused by the new frame writer.
Going with a gut feeling that the addition of a new INI file could be the problem rather than expanding existing files, I removed the new H1EDCU_GDS.ini file (which contains the IR epics channel) and restarted the DAQ. The problem went away.
To prove that the problem is with the new file rather than the total number of slow channels, I added the SENSMON channel to the H1EDCU_DAQ.ini and restarted the DAQ. The data corruption problem was still absent, and we are trending the IR.
At this point we stopped making DAQ changes and went to the code. It appears that in the file daqd.cc if an INI file's name ends with the characters "GDS.ini" the code for frame builders skips a block of code. So it appears I was unlucky in choosing the name H1EDCU_GDS.ini.
To test this, next week we will create the new file but call it H1EDCU_DMT.ini to get around the naming restriction.
Removal of Partially Loaded Filters
Hugh, Dave:
several models had accumulated partial filter loads over the past week. I investigated why and then performed a full load on:
h1asc, h1calcs, h1oaf, h1susbs, h1susetmx, h1susitmx, h1susitmy, h1susprm
SUS front end ADC/DAC replacements
Dave, Jim, Richard:
h1sush56 and h1susauxh56 systems were completely powered down. h1sush56 had its first 18bit DAC card replaced. h1susauxh56 had an ADC and interface card replaced.
On power up we noticed that h1sush56 is showing a auto-cal failure on the third DAC. On a subsequent IOP restart the 3rd DAC failed autocal again. This card is used by SR3 M2 (last 4 chans) and SUSOMC M1 (RT and SD, first two channels).
.
