CDS Tuesday Maintenance Summary: Tuesday 28th September 2021

WP9942 EX IO Chassis Upgrade

Fil, Brandon, Daniel, Jonathan, Erik, Dave:

The V5 IO Chassis for h1susex, h1seiex and h1iscex were installed. Cards from the V4 chassis were moved over to the new one, as shown in D1301004. The small contec1616 was removed (not needed now we have the new timing card) and the cards were displaced left to permit the empty slot left of the timing card.

When the IO Chassis and front ends were powered up all three did not run correctly after the IOP model was started. Symptoms included: very slow response to keyboard, not able to log in at all, slow ipmi.

Taking h1iscex first, I modified the h1iopiscex model and removed the Contec1616 part (and its associated epics-outs). This did not fix the problem.

Erik went to EX to investigate further, we found:

the timing signal to the new timing cards was not syncing at the fanout, tracked to a mismatch in the type of Avago SFP between the fanout port and the IO Chassis timing card. This was resolved by swapping out the SFP at the fanout. This would have caused timing problems, not the computer issues we were seeing.

The registers on the timing card were not correct, containing all ones (0xffffff). We had seen this before, it is a BIOS issue. Erik upgraded the BIOS on h1[sus, sei, isc]ex from 2.1 to 2.3a and this fixed all of our computer issues.

As part of this investigation, the IRIG-B card was removed from h1iscex. Once the problem was tracked down, due to the lateness of the hour, we elected to keep the IRIG-B cards in h1susex and h1seiex.

The h1susauxex machine was accidentally powered off when the other front ends were power up, easily resolved by powering it back up.

The DAQ xmit on h1oaf1 was disabled during the upgrade and re-enabled at its conclusion to prevent an accidental DAQ restart.

The SEI AI chassis were powered down throughout the upgrade and powered back up after the system was stable.

WP9941 CDS NTP Server Reset

Erik, Dave:

Erik was able to reset our NTP Server (Symmetricom Syncserver S350) back to its factory defaults via the front panel. After resetting the password, he was able to verify this unit can operate as a PTP Grandmaster. This service is only available on the second ethernet port, which we had been using for the VAC-LAN. Erik moved this over to the third ethernet port.

ethernet port	Network	IP
1	CDS-SERVER LAN	10.20.0.6/24
2	reserved for PTP	TBD
3	VAC LAN	10.1.240.1/16

Beckhoff Timing Test Setup

Daniel, Patrick, Erik, Dave:

The spare Beckhoff computer in the MSR rack 8 was connected to the CDS slow controls LAN in preparation for timing tests. It is connected to sw-msr-h1aux port 28, the switch configuration was updated accordingly. This computer has a LIGO timing card installed, it was connected to the MSR Timing Master port 8. When the Beckhoff PTP terminal is delivered this system will be connected to the NTP server PTP port and the timing testing can start.

DAQ Restart

Jonathan, Erik, Dave:

The DAQ was restarted for two updates:

• the modified IOP models for EX (h1iopsusex, h1iopseiex, h1iopiscex) required a DAQ restart (slow channels were removed)
• new EDC ini files for FEC, DAQ and SDF which did not make it in last week.

This was a h1edc-reset DAQ restart.