WP13433 TW1 offload
The first part of the offload was done. The past 6 months of data was moved to a static directory and NDS1 was temporarily reconfigured to serve this data as a separate source. This went into effect when NDS1 was restarted as part of today's DAQ restart.
WP13442 ISI HAM2,3 model changes
Jim made wiring fixes to h1isiham[2,3]. These models were installed. No DAQ restart was needed.
WP13443 Add CRS Fast Channel
A new h1crsproc model was installed to add a SUM DQ channel at 512Hz. DAQ restart was required.
WP13436 Pre-upgrade of SUSAUX and PEMMID
All the sus-aux and pem-mid front ends were upgraded to the new RCG (5.6.5) and booting Deb13 from the new bootserver (h1boot5-6). This added slow channels to all models, ADC/DAC overflow counters to all, ADC-temp to iocs. For all but h1susauxb13 the upgrade had no problems. With h1susauxb13 the EDC had issues. Following the first upgrade both DAQ legs saw continuous CRC errors from the new EDC. The error rates were usually in the 2-6 per second (out of 16 transmits per second). Initially thought of as a timing problem, we attempted to switch the EDC timing source from the IOP model to direct read of the timing card. This produed times which looked more like unix time than GPS.
After trying some things we decided to revert h1susauxb13 to deb11/rcg5.52. We then restarted the DAQ 0-leg for the susaux and pemmid model changes and h1crsproc.
We were then in a holding pattern for the 1-leg DAQ restart. While waiting, Erik found that the EDC has a large send delay in its cps-transmit. It was 34mS here compared with 5mS at LLO.
So for a test we upgraded h1susauxb13 back to deb13/rcg-5.6.5, with cps-xmit delay reduced from 34mS to 5mS. This did not fix the CRC and the error rate was unchanged.
Next Erik changed the EDC's send latency from +10mS to -10mS. This fixed the issue, even though code comments suggested otherwise.
Finally the cps-transmit was returned to 34mS, keeping the EDC at -10mS and the problem did not return. We decided to keep with this configuration.
DAQ Restarts.
As mentioned above, we had several DAQ restarts.
The first at 11:08 was 0-leg only, the EDC has been returned to 5.5.2
The second at 13:40 was 0-leg again (EDC once again upgraded) and 1-leg
EDC was not restarted for a new INI file, just for the OS/RGC change.
Because of the EDC problems found, we did not install a new H1EPICS_GRD.ini with JAC_HEATER added today. We will schedule that for tomorrow.
Overview shows frontends we have upgraded to Deb13/RCG-5.6.5 with dark orange tags.