J. Kissel We've discovered that the H1 SUS MC2 compensation filters are incorrect for the M2 stage coil driver, having never been changed after the analog coil driver was swapped out for a beefier drive (see LHO aLOG 9956). It turns out, by lucky coincidence, we've been operating the new coil driver in a state (State 1) where it's frequency response is flat, and the compensation filters -- though wrong -- cancel each other to also be flat, so the overall crossover loop gains remained unaffected. Just to be sure, however, I too a "pre" measurement of each crossover of the IMC_L loop gains. I attach the results, and summarize below. Crossover UGF [Hz] Phase Margin [deg] MC_L/MC_F 14.5 39 M2/M3 7 38 M1/M2 0.125 36 These numbers are comparable to LLO's numbers (see, e.g. LLO aLOG 4331). Templates can be found here: /ligo/svncommon/SusSVN/sus/trunk/HSTS/H1/MC2/Common/Data/ 2014-02-11_H1IMC_M1-M2_Crossover_OLGTF.xml 2014-02-11_H1IMC_M2-M3_Crossover_OLGTF.xml 2014-02-11_H1IMC_MCL-MCF_Crossover_OLGTF.xml
For posterity, the IMC loop had the following in its path:
Input Mode Cleaner, Common Mode Servo Board Settings
Common Path
- H1:IMC-REFL_SERVO_IN1GAIN = 9 [dB]
- p:z 40Hz:4kHz compensation filter ON
- p:z 1kHz:20kHz first stage boost ON (second and third stage boosts OFF)
- p:z 4kHz:17kHz filter ON
Fast Path
- H1:IMC-REFL_SERVO_FASTPOL = +
- H1:IMC-REFL_SERVO_FASTGAIN = -6 [dB]
- p:z 140kHz:70kHz highpass ON
Slow Path
- p:z [100 100]:[10 10] "generic filter"
LSC Model Settings
IMC_L Filter
- FM1; antiWhite = p:z [10 10]:[100 100] (compensates for "generic filter")
Gain = 1.0
LSC-MC Filter
- All filters OFF (when locked)
Gain = 1.0
SUS MC2 Model Settings
All coil driver states set to 1.
M3_ISCINF
- FM 6; notch_R3 = notch at highest roll mode of MC2
- FM 7; notch_V3 = notch at highest vertical mode of MC2
Gain = 1.0
M3_LOCK
- FM 3; z40:p140
- FM 9; CLP300 = Chebychev low pass at 300 [Hz]
Gain = -300
M2_LOCK
- FM3; z1:p0.1
- FM4; z1:p100
- FM10; ELP70 = elliptic low pass at 70 [Hz]
Gain = 0.1
M1_LOCK
- FM1; z0.01:p0
- FM2; z:p0.01,0.1
Gain = 1.0
I repeated the MCL-MCF crossover measurement that Jeff made since I could not get it to agree with my model. I made this measurement with the SR785 by injecting a source into excitation B through the slow path of the MC common mode board and looking at the ratio of test 1 and test 2. The results still differ from my model (likely something missing in the model..), but more suprisingly I found that this measurement showed a factor of 2 greater in magntiude in comparison to Jeff's open loop transfer function. See attached picture.
I caused a HEPI trip this evening, messing with arm cavity slow feedback to HEPI.
While trying to restet this, I once again became annoyed by how the isolate script sets the current setpoint to something other than the target setpoint for Ry and RZ. Forgetting this is part of why it took us several hours to recover from the hepi pump trip this afternoon, and it seems like by continuing to have the script do this we are shoting ourselves in the foot. I have heard that the script does this because of fear that HEPI can't handle these large offsets, but we have been using these large target positions for more than a month, so it seems HEPI can handle them. If that's true, is it possible to fix this script?
Having a closer look at the offsets, I noticed that the location was not coming back to the set point even after waiting a long time. I attached a plot of the target location over the last 60 days, it has really never been at the set point.
AS far as I understood, this after noon after our long struggle to get ETMX back after the hepi pump trip, Jim got it back one DOF at a time using 250mHz blends on stage 1, Tcrappy on stage 2, and using the 60 Hz notch in the stage 2 controllers (which is for level 3 but while we are still using level 2 controlers we need to engage FM5 by hand). (correct me if I'm wrong Jim)
Instructions for what worked for me now are:
HEPI:
untrip watchdog
Comands 2, Isolate level 1
Important! Check that DC bias current setpoints match the target setpoints.
ISI:
Set T240 watchdog thresholds to something large
Set stage 1 blends to 250, isolate level 3.
Wait a long time, get distracted by other work, come back in a half hour....
change stage 1 blends to Tcrappy
Isolate stage 2 level2 with Tcrappy
All set!
For the record, it's not that *HEPI* can't handle the large offsets, it's that the Stage 1 T240s on the BSC-ISI can't handle the large offset without saturating, and therefore tripping the ISI's watchdog. The seismic group has heard this complaint loud and clear and is considering if they even need to include the T240s in the watchdog system. We promise we're trying to make it better!
I forgot to say, the reason I was messing with the feedback from ISC to HEPI in the first place was that Brett and I noticed while he was making his measurement that the tidal relieve servo was unstable....
I will try to fix this in the morning.
Like Jeff said, we limit the transfer of the Target of HEPI in the Current Setpoint to 500 nrad to prevent saturating the ISI's T240s. One has then to copy the targets into the current setpoint manually if he wants to override this. We could consider removing this limit if we increased the T240 WD thresholds on the ISI at the beginning of the turn on process (starting with HEPI), and had the T240s removed from the blend when turning the ISI on.
Sheila mentioned another interesting behaviour that we noticed yesterday: even when the Target and Current Setpoint agree, the position on RY does not go all the way to the setpoint value along RY on ETMX. (See attached plot, and Cart Bias MEDM screen). It is not the case on ITMY and ITMX HEPI, but none of them request such high target values (see attached Cart Bias MEDM sceeens for HEPI ITMX and ITMY). One could argue that parts of HEPI may start contacting (each HEPI has a slightly different Range Of Motion which depends on how it was set up) and prevent the servo loops from pushing HEPI to the requested Current Setpoint. This would probably make the position loops go unstable though, and it is not the case. As a side-test, we increased the value of the current setpoint on RY to see if the location would follow and it does, which indicates that HEPI is not stuck contacting.
This oddity has been going on for 60 days, but has also remained consistent throughout, even though HEPI, and its pumps, got restarted more than once. HEPI alignment along RY, once controlled, has not changed for those 60 days.
As far as having trouble turning the ISI on, the shift between the floating, and requested controlled position are quite high. We tried making this smaller, which made the ISI turn on way easier, and request the needed alignment to SUS, which seems to have worked. Should we do it this way from now on, for the sake of making the ISI turn on process more repeatable on ETMX?
The HAM-ISI models, scripts and screens were successfully updated.
The update was tested on HAM2-ISI, before being ported to the other platforms. Test performed are:
Foton files had to be updated on HAM2 and HAM3 so the level 3 controllers would be in the correct filter banks for the updated command scripts to work. Filters were not modifed.
The content of the update is detailed in DCC T140012. See the attached Update_Log.txt For more details about the sequence of the update at LHO.
Work was performed under WP #4434, which is now closed
Restarted the DAQ to support Hugo's HAM ISI model changes and incorporate Daniel's latest X1PLC2 channel list.
Installed the following electronics at EX per D1001423-v18. D1002559 ISC Whitening Chassis for WFS_A D0902796 WFS I/Q Demodulator WFS_A D1100905 Demodulator Patch Panel D0902796 WFS I/Q Demodulator WFS_A D1002559 ISC Whitening Chassis for WFS_B D1100749 Legacy iLIGO WFS PD Interface Items still needed for tomorrow: 1. Power up electronics 2. Finish running long field cables to 384 Channel Acromag Binary and AA chassis 3. RF cabling from field rack to ISCTEX table Filiberto Clara
Modifications to the slow controls system were made to add WFS whitening channels and WFS demodulator readbacks.
I loaded the Tcrappy blend filters on the BS ISI today, and reworked some level 3 controllers with a more aggressive boost (while preserving the .m-file with the old working controllers!), but it's not fully working yet. I can get st1 on the new controllers, but it trips as soon as I turn any of the boosts on. For now I'm leaving the ISI on the old level 2 controller. I'll look at it again tomorrow.
I've moved IOHT2R into its proper position. The plan for tomorrow will be to transition the LVEA in the afternoon while the red light working group has PRM aligned to begin aligning the table to the SM2 trans beams.
A crashed ops machine ate my homework, so this isn't very complete...
Earthquake this morning tripped ITMX and ETMX, and others. ETMX ISI didn't come back up until 1300. There has been elevated microseism all day due to high winds.
Fil at Ex, mid-Y, Ex again
Fire Dept at EY 0900
HEPI pump station down at ex this morning, hugh down to diagnose 1100
MarkB working on tip-tilt filters 1300?
JeffB at ham4 1330
Mitch & Justin ACB cleanroom 1330
JoeD h2 psl enclosure 1200-1300
JeffL in LVEA ~1300
Me loading new isolation lvl3 filters in the BS ISI 1515
Calibration gain for MC1 MC2 MC3 alignment offsets was modified after Paul's measurement :
| OPTIC | DOF | Initial factor (cts/urad) calculated from alog 4763 | Scaling factor from alog 9870 | New gain (cts/urad) |
| MC1 | Pitch | 1.875 | 0.7043 | 1.3206 |
| Yaw | 2.681 | 0.8223 | 2.2046 | |
| MC2 | Pitch | 1.875 | 0.8572 | 1.6072 |
| Yaw | 2.681 | 0.8326 | 2.2322 | |
| MC3 | Pitch | 1.875 | 0.7823 | 1.4668 |
| Yaw | 2.681 | 0.8588 | 2.3024 |
(Alexa, Sheila)
Today we had trouble locking the green arm...
Starting to update the models, scripts and screens for HAM-ISI. Work is performed under WP #4434
Details about the update can be found in DCC T1400012
Yesterday with completion of TMS Alignment, may have jumped the gun with securing the assembly. Forgot that SEI will want to float, balance, and test the ISI. So the EQ Stop Tooling (aka Alignment Stabilization Tool) needed to be removed. Before removing it, the Swing Stop Cart was installed, and then the EQ Stops were removed. So now the TMS is ready for the ISI to be tested. So with SUS's work this afternoon and SEI's work for the next few days, that would put us back on deck to secure the TMS next week most likely.
TMS TO DO List Post-SEI Test Stand Work:
I swapped in a new HTTS filter file with corrected dewhitening filters (what claimed to be 10:0.4 had actually been 40:1.6). I checked that RM1 and RM2 still damp stably.
I've started on a round of undamped TFs on RM1 and RM2 to check that I get the expected results without post-hoc corrections.
The TFs were taken successfully. The raw data was processed with meas.badFilt = '' (no post-hoc filter correction) and meas.E1201027 = true (coil driver resistors replaced):
^/trunk/HTTS/H1/RM1/SAGM1/Results/2014-02-12_1330_H1SUSRM1_M1.mat
^/trunk/HTTS/H1/RM2/SAGM1/Results/2014-02-12_1330_H1SUSRM2_M1.mat
These data sets are plotted in the attached PDF with the latest L1:RM1 and L1:RM2 results and the immediately previous H1:RM1 and H1:RM2 results of 12/20/13. As expected, the new results without post-hoc filter correction agree with the earlier results with correction, showing that the new H1SUSHTTS.txt file is good. (It has been committed.)
The agreement with the model is good for H1:RM1 so this new data (2014-02-12_1330) can count towards Phase3b testing. The RM2 data shows the same strong coupling of L into Y as the last few measurements, and this needs to be investigated at the next vent.
LLO should impement the same fix as soon as convenient. Specifically, in L1SUSHTTS.txt, all the ???_M1_OSEMINF_?? modules should have Section 1 set to have label '10:0.4' (probably that way already) and filter zpk([10],[0.4],1,"n") (will have been zpk([40],[1.6],1,"n")).
I created a new version of the HTTS TF plotting script, plotHTTS_ddtfs_M1.m, to correct for the fact that the wrong dewhitening filter has been installed at both sites, and that the LHO ISC HAM-A coil drivers have only recently been changed per E1201027. Setting meas.badFilt = '40:1.6' (rather than ' ' or anything else) enables the correction for the bad filter, and meas.E1201027=true enables the new gains.
The actual hardware whitening filter is 0.4:10 per D1002818/D080276. The dewhitening filter installed in Slot 1 of the the OSEMINF blocks claims to be 10:0.4 but has actually been 40:1.6.
After checking with Stuart to ensure that there was no LLO data that hadn't been committed, I reran plotHTTS_ddtfs_M1.m on all the data sets listed in measList in plotallhtts_tfs_M1.m, so as to regenerate the summary .mat files and plots at ^/trunk/HTTS/ifo/optic/SAGM1/Results .
I used meas.badFilt = '40:1.6' in all cases and meas.E1201027=true for all except the measurements of H1:RM1 and H1:RM2 in 11/2013, which were before the LHO resistor change. I lost a day to head-scratching before I realized that the L1 resistors had been changed shortly after the ECR was approved in 12/2012, whereas the LHO ones were changed almost exactly a year later (12/2/13, LHO alog 8792).
As might be hoped, the new plots were free of the hump caused by the misalignment of poles and zeroes between hardware and software. The new files have been committed to the SVN.
The two attached comparison plots show L1:RM1, L1RM2, H1:RM1 and H2:RM2 before (2013-11-25) and after (2014-02-12) the fixes. Note that the H1 data in the second plot is from later (12/20/13), and fixes a second filter issue that has not been identified but only affects the 11/24/13 data).
I prepared a new filter file for LHO with the correct dewhitening filters, /opt/rtcds/userapps/release/sus/h1/filterfiles/H1SUSHTTS_dewhitefix.txt , and will install it at the earliest convenient opportunity.
I have updated the CDS MEDM overview screens: added new h1lscaux model changed the WD colors to support HIFOX
Fluid level trip. Found the Pump Controller (not servo) red lite off--other than power failure, sure indication of Fluid Level trip.
Turn on Process:
Since I've hardwired the FWD button, the controller will start at the Servos Command as soon as the Green Start Button is pushed, so:
Must turn down the Servo Output--On Pump Controller medm set servo set point to zero or manually run down the output.
Press Green Start button on Controller Cabinet, Ramp up servo set point to 80psi, or step up output manually to near 80 and then switch servo to Auto mode.
Warning--Don't overshoot the pressure too much or the level may trip again or if you really overshoot and the level doesn't trip, the pressure relief valve will release.
After I got the servo running again, I pulled up the trip switch and found the trip level was only ~1/16" below the run level. Of course to do this test you have to trip the switch and start all over again. I then lowered the switch ~1/4" so we should be good for a while.
I have a scale on the site glass and a white board logging the running levels and now I have a trip level. We have been trending down a bit over the past few months. I have a couple minor leaks at the pump station, maybe something like a teaspoon a month. That combined with evaporation and the large servo load (really minor affect I believe) led to the trip. I'll start a regular maintenance routine of noting the running fluid level and lowering the trip switch to keep ahead of it.
Patrick T., Sheila D. We went to check on the H1 PSL environment. We verified that all the HEPA fans are off. We couldn't tell if the make up air fan was running, so we commanded it to run again. It was and still is set at 20% fan speed.
plots attched.
For the ITM it was basically a one click recovery (using Tcrappy blends and isolate level 3), I tripped the ETM trying to bring it up (I tried 750 blends and level 3 on stage 1, that didn't work).
Hugh saw that the biases on the EMT ISI were fairly small, and brough it to stage 1 level 3 with T crappy, stage 2 level 2 Tcrappy.
The microseism has been rising over the last 20 hours, and there was a big spike somewhere around 2 am. (we are currently at around 5 on the strip chart, I'm not sure what the units are).
The tripping may be due to the M6.9 earthquake in China (see LLO aLOG entry ) which occurred at about 3:20 AM CST (or 1:20 AM PST). It tripped all our watchdogs
(Alexa, Hugo)
ETMX watch dog tripped at 9:45am. Similar spikes seen in ISI.
What happened at 9:44am PCT: Fluid pressure drops on HEPI (No fluid pressure on MEDM screen of HEPI pump. reason still unknown). HEPI drive increases to meet targets, but hepi does not get physically driven due to the abscence of fluid pressure. The servo increase the output drive to compensate for it to the point where HEPI trips (1). Once tripped, HEPI slowly (hydrolic system) comes back to its floating position. While slowly coming back, it saturates the T240s, which are included in the blend which is fed to the ISO, causing the ISO of the ISI to ramp up as well, and trip the ISI (2), 2 seconds after HEPI tripped.
Trip times:
HEPI: 1076262292 (1)
ISI: 1076262294 (2)
[Stefan, Lisa, Evan, Kiwamu]
Our goal tonight was to get the IR transmitted light centered on the transmon QPDs. A coarse alignment was established. We still need to do a fine alignment with the picomotors.
QPD centering:
We had never aligned this path with the actual IR beam, and neither QPDs had a beam on them. As discussed and suggested in the last integration meeting, we started this mission from a scan of the TMS in order to find an IR beam with the QPDs. Moving the TMSX in an almost random way, we found a beam which was flashing as the carrier light resonated in the cavity. The yaw angle needed to be twisted by roughly 100 urad to get the beam on the QPDs. Then, keeping the flashing beam on the QPD by steering the picomotors (M4 and M14), we steered the TMSX back to the nominal angle. This operation was successful. Then we moved onto a fine adjustment by locking the main infrared light to the arm cavity (the detail of the locking procedure can be found in alog 9644).
The fine tuning was very frustrating. We kept loosing the lock because of the roll mode of MC2 at 40 Hz (see awiki for HSTS resonances ) which then saturates everything and breaks the lock. Plus the spot position on the two QPDs didn't seem to be conversing by moving a combination of the two picomotors. The spot behaved as if the picomotors are degenerate. It is possible that, since the time for tweaking the picotomors was limited by the stability of the locking, we confused ourselves in the rush. This needs to be revisited. Anyway, a good news is that we now have a beam on both the QPDs and therefore we can do the fine adjustment anytime when we get a chance. Also we did an estimation of the power on the QPD, because we needed to clarify if the beam we saw was the main beam or a ghost. This is summarized in Evan's log (see alog 1006).
Some other things:
Later tonight, I tried to engage the AO path. Though I haven't succeeded. The idea is to have a low cross over frequency which then allows us to (re)insert the notch filter for the 40 Hz roll mode. However the loop was not so stable and everytime when I changed the AO path gain in the common mode servo board it simply broke the lock even if the gain seemed small enough. In addition to it, it became harder to lock approximately after 1:30 am probably due to some angular fluctuation in the arm cavity. I am too lazy to identify what optic is moving, but it is also clearly visible in the green ISCT1 CCD camera. Because of that, the IR locking doesn't stay more than 2 seconds -- the cavity build up in infrared drops extremely quickly due to the angular fluctuation.
End Station work:
Also, Lisa and Stefan went to the end X station to realign the single trans PD because the beam almost completely fell off from the diode during our picomotor adjustment. Because this PD was used for triggering the infrared locking, we needed to realign it. The alignment wasn't completed because the infrared light was not bright enough to easily work with. Then, it turned out that the bottom periscope mirror for the IR trans path was found to be very low reflective for some reason. It can be a wrong mirror. Indeed, Lisa and Stefan could see a beam with a card before it hits the bottom periscope mirror.
This mirror needs to be checked and possibly replaced by a proper one.
Just for reference the picomotors are now at: M4 X=-3273 Y=1796 M14 X=-1630 Y=-3515 In case any one wants to go back to approximately the position they were at before.
Also noted in alog 9731, most of the light the "IR trans" path has about 30-40 uW of green light, this may be what is transmitted by the lower persicope mirror.