I logged into H1HWSEY and restarted the EPICS IOC. A new channel (H1:TCS-ETMY_HWS_FREE_DISK_FRAC) was added in v1.1.8 of the HWS code and the IOC had to be restarted to incorporate it.
I was asked to add to the SUS Guardian the ability to turn off the DarmDamp filter block outputs when the suspension goes into SAFE and turn them back on when the other outputs are turned back on (ENGAGE_DAMPING). To do this I added three methods in sustools2.py in the same style as the rest of the methods for filter blocks, and the appropriate dictionary values to susData. These filter banks are located on the quads in the M0 stage and act on the V,R degrees of freedom.
Added methods are:
# Methods for DARM_DAMP blocks
def darmDampPvs(self, levels=[], chans=[], suffix='', verbose=False, withprefix='bare', matlab=False):
return self.levelfilterblockpvs('darmdamp', levels=levels, chans=chans, suffix=suffix, verbose=verbose, withprefix=withprefix, matlab=matlab)
def darmDampOutputSwitchWrite(self, enable, levels=[], chans=[], verbose=False, pair='none', withprefix='bare', matlab=False):
return self.genSwitchWrite(self.darmDampPvs, 'OUTPUT', enable=enable, levels=levels, chans=chans, verbose=verbose, pair=pair, withprefix=withprefix, matlab=matlab)
def darmDampOutputSwitchRead(self, levels=[], chans=[], verbose=False, pair='value', withprefix='bare', matlab=False):
return self.genSwitchRead(self.darmDampPvs, 'OUTPUT', levels=levels, chans=chans, verbose=verbose, pair=pair, withprefix=withprefix, matlab=matlab)
I tested the new code on ITMY, and SR3 just in case, and it all seems to work well. The new code is committed to the svn and loaded into all of the quad guardians.
Richard, Jim The h1susey computer will be unusable for a while as we investigate 18 bit DAC problems. Initial investigation shows that D4 now fails to do it's auto calibration, where it initially succeeded when the computer was first started.
Richard replaced DAC D0, Kiwamu tested the problem channel and the effect noted in alog 18569 is not present.
Jim W.: work on FF and blend filters on HAMs 4 and 5 Hugh: possible check of vertical offset on BS HEPI Richard, Jim B: investigate problem with ETMY SUS DAC Richard: investigate ETMX ESD drive Jason: PSL alignment Sudarshan: end Y PCAL calibration Apollo: beam tube cleaning
Peter F, Evan, Kiwamu,
ETMY was not damping today. It sometimes damps OK and sometimes does not. When it does not damp, it grows an oscillation in pitch at 0.5 Hz. Also, steering the suspension often kicked the suspension for some reason. After many hours of investigation, we finaly found that the DAC outputs show some discrete steps in every ~ 2000 counts or so. This, of course, made the actuators highly nonlinear. This was very sneaky -- one would not notice until he or she crossed a discrete point because the actuators are locally linear. The attached is a screen shot of StripTool showing the nonlinearlity. The purple curve is the requested DAC counts and the yellow one is an actual analog voltage observed by VOLTMON. As seen in the screen shot, the analog signal has multiple discrete steps. This behavior was seen at all 6 coils on the M0 (top) stage of ETMY and very repeatable. We didn't have energy to check the lower stages and therefore we don't know if this is true at the lower stages.
We must fix this issue.
Jeff, Calum, Ben, Rich Today we got quite a bit of the controls checked out on the Y-end ESD. We verified the functionality of the following: 1. All Quadrant analog drive and readback signals associated with the HV and LV ESD are working. The readback channels for the LV ESD were verified at the channel level, but are not yet incorporated into the MEDM screens. 2. All 15 binary control and readback channels are functional We ran into trouble on the Bias drive channel, which we troubleshot down to the DAC card. A new DAC card (SUS DAC 4) will have to be installed to make this functional. We also need to finish taking the transfer functions of the installed system. Jeff has left us specific instructions for doing that in the morning. We had 3 or 4 trips of both HV supplies while we were working, leading us to be suspicious of the vacuum interlocks associated with the HV ESD. This has not yet been explored much, but will prove annoying if it persists as you cannot remotely reset this particular trip. It could be associated with us relocating the interlock chassis within the SUS field rack, but the chassis in question seems simple enough to have tolerated a small move. Here is a summary for the use of the manual control test fixture (D1500130) associated with the LV ESD amplifier: Function -> State -> Meaning HVLV Switch -> LED Lit -> The output drive mode is selected for LV operation QuadrantSelect -> LED Lit -> PI Drive aligned to UR or UL HV ESD ON/OFF -> LED Lit -> Not in reset or toggle mode ie. nothing happening in reset land Input Relay -> LED Lit -> Input quadrant drive signal is not connected to the HV ESD amplifier PZ Bypass -> LED Lit -> The associated Pole-Zero stage in the LV quadrant drive chain is bypassed
The Vacuum interlock had been working quite reliably since installation. I would suspect something must have become loose after the interface chassis was moved the other day. Not sure what alog entry that was noted in. I will open up the unit to check all connections.
Cheryl, Nutsinee
07:00 Richard in and out number of times CER
08:17 Richard taking all SUS in HAM2 to SAFE. Fixing MC3 issue.
9:12 HAM2 SUS restored. IMC locking.
10:30 Jeff B. to cleaning area
10:45 Jeff back
11:50 Rick & Nutsinee to LVEA
12:06 Rick & Nutsinee back
12:33 Jeff K. restarts ETMY SUS
15:00 Elli to LVEA (HWS table)
16:00 Nutsinee to LVEA (HWS table)
18:20 Nutsinee & Elli out.
SUS ETMY is not happy today.
Nutsinee, Elli
We ahve aligned ITMY HWS so that it gets a sled return beam. Attached is a photo from the HWS with no Hartmann plate on. The alignment was done with the X-arm locked. The beam looks clipped, but we spent a few hours today looking for clipping in both the outgoing sled beam and in the incoming ALS beam and this is the cleanest image we could get. We made sure both beams are centered on all optics. I have put the Hartmann plate on the camera, so that we can try to get a measurement using the HWS camera.
On Monday & Tuesday the operator had to respond to many HEPI Pressure alarms. They did not appear real to me as they seem to be glitches and steps in the pressure, things that really don't happen. At the time I associated it with all the electronics changes EE was doing. I surmised the powering off and on as things were replaced was causing ground state shifts on the pump station controller in turn changing the pressure transducers response or offsets.
Checked the HEPI Platforms and they were off during the pressure transients so it was not the platforms de-isolating or re-isolating. We'd have seen this before if it was that.
Attached are a few trend plots, but nothing is definitive.
The first attachment is the corner station. I'm plotting the return and supply pressures at BSC2, the difference between them for the servo set point. The VOUT controls the motor/pump speed and the last trace is the MAINSMON, something to do with the building power supply. A few glitches occurred before lunch. But at 1325 it appears the Return Pressure droops more than Supply increasing the differential pressure and the servo drops the speed of the pump to keep the DP correct and this continues down and then back up in a series of steps. Further after lunch more steps are seen. Interesting to see the MAINSMON go flat for a few hours (the data is there) and then glitch back to looking live just as the final pressure jump puts everything back to normal at 1740 on Tuesday.
The second attachment is for EndY but showing you fewer channels and including the HPI OUTF to show that the platform isolation is not the cause. This story for EndY is similar to the corner station except it is pretty much a single step in the pressures and motor speed. Again, everything jumps back to normal when the MainsMon comes back alive, hmm at 1740pdt.
For EndX, this occurred on Monday. The picture isn't as clear (3rd attachment,) the sensors are much noisier at EndX and the MainsMon came back on a couple times. The first step also occurs about the time Ken the electrician is at EndX for access system work (alog 18496.) There is also correlation at one occurrence of the MainsMon coming back alive.
Can't say exactly what was causing these pressure glitches but the pump station really responds to them and had the platform been isolating, it would have been seen on them as the true differential pressure across the HEPI Actuator would change.
Bottom line, when Richard tells you it won't bother you or it isn't his fault, grab your salt shaker!
Sudarshan, Kiwamu, Rich, Peter F, Jeff, Jim, Dave:
h1psliss was updated from Duncan's latest svn version, which makes the ODC channel an 32 bit unsigned interger DAQ data type (7)
[H1:PSL-ODC_CHANNEL_OUT_DQ]
acquire=3
datarate=16384
datatype=7
Model changes were made to h1susetmy and h1sustmsy.
The routing of the Duotone signal out of the last channel of the first DAC on h1iopsusey was turned off.
The DAQ was restarted to sync it back up the the PSL and SUS ETMY changes.
- continuation of hardware upgrade, things starting to come back, like PSL
- starting to relock - X arm went ok, Y arm was still not locking at 3PM
- ETMY got new code, JeffK and others looking into that optics issues, i.e. arm not locking
- Nutsinee, Cheryl
The purge air system was turned off at the end of the day today.
Every ISI and all HAM HEPIs except HAM4 SDF had channels not found in their list. From 28 to 1476 not found channels. These were all alarm fields and related to old guarding channels, Sensor Correction scheme changes, and Oplev channels going away. A total of 7192 lines to remove from the safe_alarms.snap files, see 18275. Hopefully, this is the last of these issues popping up in SDF.
Elli, Aidan
HWS code v1.1.9 is installed on hws computers h1hwsmsr, h1hwsex and h1hwsey. Updated symlink to HWS directory. Changed aliases for the Run_HWS_H!I/ETMX/Y command. (Refer to LLO alog 18291 for a lsit of the changes made.)
There is now a run-time error when running Run_HWS_H1ETMY and stream_image_Y on h1hwsey, and a run-time error running stream_image_X on h1hwsex, which we will work to resolve.
Filiberto, Peter, Jeff, Rich Over the last few days, we have successfully installed the Low Noise ESD Driver (D1500129) and its associated field wiring. This unit has 15 individual control lines: 4 bits are associated with switches to connect or disconnect the quadrant drive signals to the HV ESD amplifier that pass through the Low Noise Driver for control 4 bits are associated with switches to select or bypass the pole-zero stages in each quadrant drive path. 4 bits are used to select the output drive mode between High Voltage mode and Low Voltage mode for each quadrant. 2 bits are used to steer the two Parametric Instability (PI) drive inputs to any of the four available quadrants. 1 bit is used to enable or disable the High Voltage ESD amplifier A wiring diagram for the entire ESD system is preliminarily available under D1400177-v4 pending the required DCN process. Details of the Low Noise Driver circuitry are available under D1500129 The serial number of the installed chassis is S1500066. The test results from manufacture are available at this serial number. A test fixture has been supplied that allows manual operation of the low noise driver. Details of the test fixture are available under D1500130 This afternoon, Jeff and I are planning to take detailed transfer functions of the driver for better filter compensation. Jeff has also been working on the needed model updates and control screens.
This is my understanding of the new infrastructure that Jeff has installed for controlling the driver. (If anything in this entry is wrong, please correct me.)
The ETMY BIO screen has been expanded to allow for binary control of the new driver, similar to how coil switching is currently done. See attached screenshot.
"State request" controls whether the driver low-pass filtering is engaged. When engaged the driver filtering is two poles at 2.2 Hz and two zeros at 50 Hz. (See D1500016, "Pole-Zero and Driver").
Changing the state request will automatically engage or disengage digital compensation filters which are installed in FM2, FM6, and FM7 of the ESD output filter modules, just like with the coil switching. FM2 (simLP) is the simulated low-pass response (zpk([50;50],[2.2;2.2],1,"n")), and FM7 (antiLP) is the complement of FM2. FM6 (antiAcq) is just a flat gain of 1.
"Hi/lo voltage" controls whether each ESD quadrant is driven by the Strathclyde driver ("hi volts") or the circuity in the new low-voltage driver ("lo volts").
"Hi volt disconnect" controls whether the Strathclyde driver receives input from the DAC.
Per Jeff Kissel's request Betsy and I (Xavi) looked at several SYS subsystem channels to perform timing checks on the IO chassis. These checks were in slides by Shivaraj and the slides should be added to this alog and references to the slides added. The channels we looked at (shown in the attached green medm screen shot) are: H1:SYS-TIMING_C_MA_A_PORT_2_SLAVE_CFC_TIMEDIFF_1 (Master1) H1:SYS-TIMING_C_MA_A_PORT_2_SLAVE_CFC_TIMEDIFF_2 (Master2) H1:SYS-TIMING_Y_FO_A_PORT_9_SLAVE_CFC_TIMEDIFF_1 (Y-arm) H1:SYS-TIMING_X_FO_A_PORT_9_SLAVE_CFC_TIMEDIFF_1 (X-arm) Two of them are in the Master, one in the y-arm, and the third in the x-arm. It does not appear that any other channels are active at this time. We took a week of data (in the past of today) for each of these channels and attach figures that show the time series (to check for trends) as well as histograms. All in all these plots look reassuring. Notable features: 1) The timing differences between adjacent points are integer multiples of 7.45ns. Mostly the integer is 1 (see Master1TimeseriesDiff.jpg, only posted it for Master 1, others look the same). 2) There do not appear to be significant drifts (e.g. < 50ns for 1 week in Master 1) 3) The spread is about 50ns (shown in figures labeled Histograms) 4) there is ~1 day periodicity in the time series for Master 2 5) There seem to be common glitches (small ones at the level of ~0.25 us, e.g. see all time series toward the end) Betsy and Xavi
For Livingston I was able to look at the following channels: L1:SYS-TIMING_C_MA_A_PORT_2_SLAVE_CFC_TIMEDIFF_1 (Master1) L1:SYS-TIMING_C_MA_A_PORT_2_SLAVE_CFC_TIMEDIFF_2 (Master2) L1:SYS-TIMING_Y_FO_A_PORT_9_SLAVE_CFC_TIMEDIFF_1 (Y-arm) L1:SYS-TIMING_X_FO_A_PORT_9_SLAVE_CFC_TIMEDIFF_1 (X-arm) The channels exists but the time series are populated with zeros. X
The DAQ EDCU connected to this channel and is now GREEN.
Note to operations staff: during ER7 please investigate if the EDCU is not GREEN.