Camilla, Betsy, TJ, Fil, Ryan C, Richard, Erik, Dave

We had a very strange problem with the first 20bit-DAC in h1sush7, which h1susfc1 uses for its six M1 drives (T1, T2, T3, LF, RT, SD). Even though h1iopsush7 reported that DAC0 was driving correctly, there was no voltages being produced by the AI chassis.

We are in the early stages of figuring out what was wrong and how it got fixed. Due to HAM7 in-chamber urgency, once it started working we handed h1sush7 over the the HAM7 team.

Summary of what we did is:

tried various power cycles of IO Chassis and AI Chassis, no change. Put DVM onto DAC cable at rear of AI Chassis, looking at CH0 initially. No voltage. IOP correctly detected loss of AI.

With IO Chassis powered up, but computer down, Fil saw 3.5V on CH0, which went to 0V as soon as the computer was powered up. We are not sure this is normal behaviour, but we saw it on two different 20bit-DAC cards. 

Next we replaced the 20bit-DAC, no change. Old (removed) 210303-09 New (installed) 210303-14.

Erik reminded us that since this is the first DAC in the IO Chassis, we can test CH7 with the DUOTONE loop back (IOP feeds ADC0-CH31 directly into DAC0-7, switches relays on Interface cards which routes DAC0-7 into ADC0-30). We turned this on, ADC0-30 saw the DT. 

Next we replaced the DAC's ribbon cable and interface card. This make things worse, no CH0 signal and now no Duotone. Interface old (removed) S1101353, new (installed) S1000868.

Thinking that ribbon cables are more likely to fail than IF cards, we went back to the original IF card S1101353, installed another ribbon cable, and shuffled DAC0 over to the second Adnaco backplane. THIS FIXED IT.

Several changes were made here, so we are not sure if it was the slot move or the ribbon cable or both.

Here is the original card layout (not shown A4-1 = BIO0)

and here is the one we ended up with (ADCs shuffled one left in A2, 20DAC0 moved to A2-1)

We tested the first 6 channels (CH0-CH5) using h1susfc1 by untripping the watchdogs and driving the DAC with a FM offset of 10000, resulting in a 20bit-DAC drive of ~40000 counts, which gives 0.7V DAC output. CH7 was tested with the Duotone.

I was reminded that on the 1st December 2025 the h1iscex 18bit-DAC to 20bit-DAC upgrade failed with no drive on the new DAC. That was also the first DAC, and at the time we tested the DUOTONE signal (in fact DAC-DT is always on for CAL). In that DAC all the channels were not operational, and replacing the DAC fixed it immediately. So it looks like today's problem is different, and a much more insideous one because internally everything looks nominal but no voltage is actually sent by the DAC.