Ibrahim, Ryan S, EJ, Erik, Jonathan, Dave:
We chose to fix h1susb2h34's 20bit-DAC issue with the boot parameter pci=pci_bus_safe rather than a BIOS MPS=128 change. We did however make the BIOS change to Enable PERR/SERR support.
1. Control room put BS, MC2, SR2, PR2 into safe. I bypassed the seismic SWWD for BSC2, HAM3,4
2. I powered down the AI chassis which is driven by the two 20bit-DACs. It is in rack SUS-C1 U24. From front, left side is DAC0, right side is DAC1.
3. I verified the IOP shows both 20bit-DACs in AI-WD mode. I turned on DAC-DUOTONE and verified no readback on ADC0-30 (safe to do so now the AI is powered down)
4. EJ configured h1vmboot5-5 to boot h1susb2h34 with pci=pcie_bus_safe and disabled the auto-start of the models.
5. We stopped the models, fenced from Dolphin and rebooted.
6. EJ caught the reboot, went into BIOS, Enabled PERR/SERR Support and reset.
7. After the reboot, we verified the Adnaco cards had MPS=128B and the Dolphin IX had MPS=256B.
8. We started the models. I enabled DAC-DUOTONE (AI still off) and now DAC0-7 is driving correctly.
9. EJ ran his DAC-FIFO-Check, both 20bit-DACs checked out.
10. We started the local_dc and cps_xmit to get DAQ data moving again.
11. I powered the AI chassis on. Unbypassed SWWD and handed system over to the control room.
We will leave this system in no-auto-start state from now until the upgrade at the end of this month.
Instructions to start the DAQ data after models have been started (as root):
mbuf_probe list # Check models are writing
systemctl start rts-local_dc
mbu_probe list # Check local_dc shows as last item
systemctl start rts-transport@cps_xmit