This morning when I came in I found the HAM2/3 ISIs tripped. I have tracked this down to what I believe to be a DAC communication problem in the front end.
While the models show output on their DACs, the IOP shows no corresponding DAC output on the same channels. There are no error flag whatesoever to indicate that there might be a problem. None of the watchdogs on this machine are currently tripped.
The image below show the DAC_MONITOR screens for H1IOPSEIH23, which shows zero on all outputs, and H1ISIHAM3, which shows and output of -958 counts on CH 8.
The dataviewer program has been updated for Linux and Mac OS X to correct issues with leap second handling and to allow short channel names to be selected from the menu. Current version is now dv-2.8.1
Filberto, Daniel
We moved the doubler which drives the ALS fiber distribution box from rack ISC-R1 to ISC-R4. This hopefully reduces the electronics crosstalk between this unit and the PSL VCO. We als used a differemt RF patch panel in ISC-R2. The RF power levels have been re-measured, requiring to change the attenuator at the ALS fiber distribution chassis input from 1dB to 3dB.
I just spent another hour turning on ISI ETMX. (see previous elog for ITMX.) This seems to me that most of the turn-on trouble comes from our attempt to turn on loops in an equilibrium position, and afterwards move to the target position. This move then leads to DC saturations in various sensors. Question: Why don't we put in fixed drive offsets - even after the DACkill - that allow us to make the OFF position equal to the locked position. This way the ISI never has to "move", we never have T240 saturations.
When the watchdog turns things off, they really need to be off. Last time a fiber was broken, it was because there was a bogus signal coming from the IOP process. That said, We agree that it would be really good if turning the ISI on/off didn't mess with the alignment so much. One clear action item is to trip the watchdog less often - all suggestions on this are welcome. Other Suggestions - 1) Hold most offsets with HPI - we have in mind a way to hold HPI offsets longer during watchdog trips. 2) Stop going back to the target locations for most DOFs on the ISI. One can not see how recovering the old X, Y, Z, and roll DOF (e.g. if the beam direction is along X, the rX direction seems not-very-important at the micro-radian scale). Clearly yaw is important for all optics, and pitch is important for tables with TMS. 3) get the guardian for BSC-ISI up so we can turn on ISI whilst ignoring the T240s - in the mean time, this will be put into the BSCISItool scripts available in the next few days (testing is ongoing)
>When the watchdog turns things off, they really need to be off. >Last time a fiber was broken, it was because there was a bogus signal coming from the IOP process. I don't understand. A constant value is just as "off" as the value 0. Not energy will be transferred to the payload.
I've started automation work on the HAM2/3 ISIs, started at roughly 07:00 PST this morning.
While the IMC is currently locked, this work is somewhat invasive. Both ISIs are currently tripped, for instance.
I will report when the work is done.
Couldn't turn on ISI ITMX...
I added an X arm guardian today, and tested as much of it as I could without having an arm cavity to lock. This also involved some changes to the ALSEndStateMachine Library in Beckhoff (for error reporting), and changesto the guardian overview and ALS Main medm screens.
ITMY TF will be running overnight on opsws8. Started at around 20:45
Measurement completed and lasted ~10hours.
In prepartation for tomorrows guardian deployment on ISI HAM2/3, I checked the configurations of the filter modules in the HAM2/3 ISIs, to make sure they are configured as needed for the guardian. I found a couple of discrepencies that I've tried to sort out.
The BLEND configuration is not currently touched by the guardian code, so we're not working about their configuration for the moment.
The configuration of the DAMP filter modules seems to be as expected (FM1 loaded only with 'Damp').
The ISO modules were a different story. ISI_HAMX expects the following filter banks to be populated for the vaious isolation levels:
HAM2/3 has nothing in FM3/5/6/7. I therefore added unity gain filters into those banks, and gave them the appropriate names.
The Boost_* filters also didn't have the 5 second ramp times that we decided we needed during the HAM4 testing. I added those ramps as well.
I have saved the filter files, but I have not yet loaded them into HAM2/3. I will load them when I start my tests tomorrow morning.
There is a typo in the HIGH - HIGH uses FM 4,5,6, possibly 7, and boosts with FM8 (using the 1-10 numbering)
Below are a series of reference alignment values that allow hitting the various BAFFLE PDs: # TMS pointing straight shot # ITMX PD1 caput H1:SUS-ETMX_M0_OPTICALIGN_P_OFFSET 0 caput H1:SUS-ETMX_M0_OPTICALIGN_Y_OFFSET 0 caput H1:SUS-TMSX_M1_OPTICALIGN_P_OFFSET 203.3 caput H1:SUS-TMSX_M1_OPTICALIGN_Y_OFFSET -228.1 # TMS pointing straight shot # ITMX PD3 (PD4) caput H1:SUS-ETMX_M0_OPTICALIGN_P_OFFSET 0 caput H1:SUS-ETMX_M0_OPTICALIGN_Y_OFFSET 0 caput H1:SUS-TMSX_M1_OPTICALIGN_P_OFFSET 272.7 caput H1:SUS-TMSX_M1_OPTICALIGN_Y_OFFSET -289.7 # ITM pointing # ETMX PD1 caput H1:SUS-ETMX_M0_OPTICALIGN_P_OFFSET 0 caput H1:SUS-ETMX_M0_OPTICALIGN_Y_OFFSET 0 caput H1:SUS-ITMX_M0_OPTICALIGN_P_OFFSET 35.3 caput H1:SUS-ITMX_M0_OPTICALIGN_Y_OFFSET -69.2 caput H1:SUS-TMSX_M1_OPTICALIGN_P_OFFSET 238.0 caput H1:SUS-TMSX_M1_OPTICALIGN_Y_OFFSET -258.9 # ITM pointing # ETMX PD4 caput H1:SUS-ETMX_M0_OPTICALIGN_P_OFFSET 0 caput H1:SUS-ETMX_M0_OPTICALIGN_Y_OFFSET 0 caput H1:SUS-ITMX_M0_OPTICALIGN_P_OFFSET 67.4 caput H1:SUS-ITMX_M0_OPTICALIGN_Y_OFFSET -36.3 caput H1:SUS-TMSX_M1_OPTICALIGN_P_OFFSET 238.0 caput H1:SUS-TMSX_M1_OPTICALIGN_Y_OFFSET -258.9 # ETM pointing # ITMX PD1 caput H1:SUS-ETMX_M0_OPTICALIGN_P_OFFSET 231.4 caput H1:SUS-ETMX_M0_OPTICALIGN_Y_OFFSET 90.8 caput H1:SUS-ITMX_M0_OPTICALIGN_P_OFFSET 51.35 caput H1:SUS-ITMX_M0_OPTICALIGN_Y_OFFSET -52.75 caput H1:SUS-TMSX_M1_OPTICALIGN_P_OFFSET 238.0 caput H1:SUS-TMSX_M1_OPTICALIGN_Y_OFFSET -258.9 # ETM pointing # ITMX PD3 (PD4) caput H1:SUS-ETMX_M0_OPTICALIGN_P_OFFSET 255.4 caput H1:SUS-ETMX_M0_OPTICALIGN_Y_OFFSET 60.7 caput H1:SUS-ITMX_M0_OPTICALIGN_P_OFFSET 51.35 caput H1:SUS-ITMX_M0_OPTICALIGN_Y_OFFSET -52.75 caput H1:SUS-TMSX_M1_OPTICALIGN_P_OFFSET 238.0 caput H1:SUS-TMSX_M1_OPTICALIGN_Y_OFFSET -258.9
Fabrice and Dave.
We completed the build, install and restart of the BSC ISI systems using the older version of the WD code (minus HWWD and reset inputs).
h1isi[bs,itmx,itmy,etmx,etmy] were restarted. Fabrice verified the WD code is running correctly.
I verified the safe.snap restores went without any errors. I cleared the IPC errors caused by the restarts. No DAQ restart is required.
9:00 Kris to EX, back at noon
9:30 HughR, MitchR to HAM4, done at 10:30
10:00 Kiwamu PR2 Model changes, multiple DAQ restarts
10:00 ETMX ISI & ITMX ISI and HPI found to be down. ETMX came back up, ITMX HPI would not. ITMX HPI has been down since yesterday's boot fest. Looks like the HPI IPS targets got lost during a burt restore. This was not recovered until the afternoon. It required finding an older burt file, then redoing the safe.snap.
10:15 CoreyG to EX TMS lab. Travis to LVEA SUS test stand.
10:30 HughR, MitchR to EY for cabling. Done at 12
11:00 I went to HAM4 for a fruitless troubleshooting mission. Problems fix themselves.
11:15 DaveB restarting BS-ISI.
12:30 Cleaning crew to EY. Done 15:00
13:00 Ed and Evan to IOT2R. Transitioning to laser hazard.
14:00 CraigC, RIckS, ChrisS to H2 PSL.
14:00 JeffB, AndresR to HAMs 2,4&5 hunting for install hardware. Done 15:00
Replaced the incorrect feed thrus with the needed 3-Pin power style. Then completed the cable attachment, cable strain relief and finally the easier half of the Dial Indicators. Almost that is--I was short three 5" bolts to strain relieve some SUS cables on the South side. I must have cut myself a little to close wrt inventory. But generally, all the really needed dirty work is done.
The Apollo crew got Test Stand, extra Spiral stair case, 3P spreader, and a few other things from the VEA.
The cleaning crew jumped right on as soon as I was out of the way--a few dust alarms with that work.
Crud--I forgot the CPS sync cables...Will be routed on the South & West side--away from the open door.
Fabrice's sensor correction for ETMX stage 2 was just turned back on (in the last few minutes). I did this by turning on the output of H1:ISI-ETMX_ST2_SENSCOR_X_MATCH and setting the gain H1:ISI-ETMX_ST2_SENSCOR_X_MATCH_GAIN=1.15. This helped the pitch motion.
In the attached screenshot you can see that it clearly helped to reduce the pitch motion.
The 17 V power to the ALS fiber distribution box was loose, I'm not sure if this happened while we were searching for the cause of the IMC wandering peak yesterday. At least the TwinCAT PLL autolocker code gave us a usefull error message, and the problem was easy to track down. They are tightened now.
Deleted 4 channels, inserted 1,405 for a current total of 123,768.
Mitchell and I installed the TMDs on the HAM ISI Blade Springs similar to as Hugo and I did on WHAM3. Mitchell has the serial numbers. There were also a couple feedthrus that needed to have the in-vac cable attachment screws secured. Mitchell did this.
Serial numbers for the Blade Spring Tuned Mass Dampers (D0900703) are 025, 026, 027.