D. Barker, J. Kissel

In order to better understand the auxiliary chassis' signals which monitor the coil driver's output "noise," voltage, current, and rms current, I've added the following H2 SUS ETMY channels to the frames:

Channel                         Sample Rate
H2:SUS-ETMY_M0_FASTIMON_SD_DQ   2048
H2:SUS-ETMY_M0_NOISEMON_SD_DQ   2048
H2:SUS-ETMY_M0_RMSIMON_SD_DQ    2048 
H2:SUS-ETMY_M0_VOLTMON_SD_DQ    2048 

H2:SUS-ETMY_R0_FASTIMON_SD_DQ   2048 
H2:SUS-ETMY_R0_NOISEMON_SD_DQ   2048 
H2:SUS-ETMY_R0_RMSIMON_SD_DQ    2048 
H2:SUS-ETMY_R0_VOLTMON_SD_DQ    2048 


Details:
----------------
Because I didn't want to break anything in the morning on a Saturday before leaving the state, I added these channels to the frames by directly modifying the
/opt/rtcds/lho/h2/chans/daq/H2SUSAUXB6.ini
file, uncommenting the above listed channels, and switching the acquire flag to 1. I followed this edit by hitting DAQ_RELOAD on the H2SUSAUXB6_GDS_TP.adl screen, with the intent to then restart the framebuilder (i.e. h2susdc0); a practice that's normal, and confirmed to work a few days ago.

HOWEVER, upon hitting DAQ_RELOAD, all H2 frontends immediate indicated a red, 0x2bad status, and the H2SUSAUXB6 front end crashed (as indicated by all white channels on Dave's H2CDSOVERVIEW.adl screen).

I called Dave immediately. Thankfully, our savior was home, up, and happy to help. Here's what we did:

I restarted the framebuilder:
     contorls$ telnet h2dc0 8087
     Trying 10.201.0.160...
     Connected to h2dc0.cds.ligo-wa.caltech.edu.
     Escape character is '^]'.
     daqd> shutdown

No dice. All frontend still show 0x2bad.

Attempted to power cycle h2iopb6 (IOP dcuid = 115, H2SUSAUXB6 = 116) remotely.
     - Open a web browser, go to URL http://10.99.201.${dcuid}
     - Select "Remote Control" tab
     - Hit "Power Control" button
     - Select "Power Cycle" radio button
     - Hit "Perform Action" button
Got no response after two attempts, realized I had accidentally went to http://10.99.201.11/ (h2iopseib6). *sigh*. Curiously enough, the h2seib6 computer didn't seem to reboot...
Went to http://10.99.201.115/, and performed the above steps.

The framebuilder status for h2iopsusauxb6 immediately flipped to 0x0 and, after quite a bit of time, the IOP model came back up as indicated by his channels going from white to live.

While I was futzing around with that (maybe during the first few attempts at rebooting "h2susb6," when I was actually rebooting h2seib6), Dave had removed the h2susauxb6 user model from [[some list, whose name/location I didn't catch; but NOT the /opt/rtcds/lho/h2/target/h2dc0/master file]], believing it was preventing the power cycle and/or causing the computer to hang on reboot.

Once we saw the h2iopsusauxb6 alive, he reinstalled the h2susauxb6 user model, and viola! the h2susauxb6 framebuilder status switched to 0x0, and after a similarly long time [[and not just because of the EPICS gateway]], his channels turned live.

Dave had hoped that once we got h2susb6 up, the remaining front ends would restore good 0x0 frame builder status, but no such luck.

Dave, having encountered this problem just this past week when Vincent added some channels to the framebuilder, he knew that he had to restart all the frontend's mx_stream process. This is done by logging into each frontend independently and running
/etc/start_streamers.sh.

One by one, after each kickstart of a given frontend's mx_stream, each model's framebuilder status returned to a pleasant 0x0.

Hazaah!!


Finally, after ~30 minutes, I let Dave get back to his weekend. Did I mention "Hazaah!!"?

The new channels are now in the frames, and I've confirmed I can receive data in the past via DTT. *Sheesh*

Note that this was supposed to be a quick, clean, unobtrusive way to temporarily get channels in the frames -- because I "didn't want to break everything by risking a model recompile." You can see how that went (*).(*). On Monday, or sometime soon, we will do the recommended new-standard method of adding channels to frames by including them in a permanent text list inside the user model, and re-compiling.