I found the 9 sec timing error in the test mass charge measurement mentioned in LHO log 15983. See UR_timing_err.pdf. The problem was in the matlab analysis script, not in the python measuring script. The measurement script saved a seperate file for the excitations for each quadrant and bias level. See for example, UR_Exc_raw_data.pdf. The excitations in these files are preceded by 9 or 10 seconds and trailed by 3 or 4 seconds where the excitation is nominally zero. The analysis script then finds the exact point where the excitation starts by searching for the first point where the excitation data is non-zero. The problem is that the very beginning of the excitation is not always exactly zero because of some tiny residual from a previous excitation. The analysis code was aware of this problem, and therefore cut out the first second. However, 1 second was not always quite enough. Sometimes a non-zero data point or two would survive the 1 sec cut. So I extended this cut to 2 seconds. So far this removes the problem from all the analysis. See UR_timing_err_fixed.pdf.
The new trend results are in ETMY_Charge_Trend_8Jan2015.pdf. The timing error fix inf;luenced the pitch results by +- 5 volts and the yaw results by +- 15 V.
All charge scripts are on the SVN in
.../SusSVN/sus/trunk/QUAD/Common/Scripts/
The charge measuring script is
ESD_UL_LL_UR_LR_charge_07-H1.py
This scripted can be run on repeat many times using the script
ChargeTest_H1.py
It is set to repeat the measurement 10 times, waiting 5 minutes between runs (currently each run is about 10 minutes long).
The Matlab analysis script that processes the results is
ESD_UL_LL_UR_LR_analysis07_H1.m
There is a script for plotting trends over multiple measurements, but it needs some work to generalize it for different suspensions and different data sets.
Long_Trend_H1_ETMY.m
In log 15983 the date stamp of the trend figure should read Jan 8, not Jan 9.
J. Kissel I've caputured and committed new safe.snap files in order to capture the updated ODC bitmasks (see LHO aLOG 15951), as well as to remove old Guardian variables that are no longer in the autoburt.req file as of the removal of the guardian block from all models. A full list of those models' snap files that have been captured is shown below. Once captured, I've made sure that the alignment offsets and ASC loops have been restored. Note that some suspensions have integrators in the top stage P and Y LOCK filters, which have stored values in them (i.e. PRM, PR3, SRM, and SR3, BS, and ITMs). In order to capture the safe, I've turned off the outputs to these filters. However, again, once captured I've restored them and ensure the same outputs. h1susim h1sushtts h1susmc1 h1susmc2 h1susmc3 h1susprm h1suspr2 h1suspr3 h1sussrm h1sussr2 h1sussr3 h1susbs h1susitmx h1susitmy h1susetmx h1susetmy h1sustmsx h1sustmsy
In order to calculate the sensor correction gain, I put HAM2 in this configuration for the night:
All DOFs:
- 750mHz blend
- Sensor correction OFF
I'll put it back to its nominal configuration Monday morning
The matching gains for X, Y and Z are
0.9750
0.8431
0.8167
Following the work done last week (see this alog), I've done some more tests today.
First the unsuccessful results:
- I've switched from a lvl3 conroller configuration to a lvl2 with no success. The issue doesn't come from the control loops.
- I turned OFF all the damping loops of suspensions (MC2 and PR2). This didn't affect at all the amplitude of the peaks: the suspensions behavior seems unrelated with our issue.
Now the good part:
The problem seems directly correlated with the blends, and especially with the blends on RX and RY. I'm making this affirmation because of 2 things:
1. The peak disappears when I switch the blends on RX and RY. It stays up otherwise. In the plot atrached, I've switched from a 01_28 blend (solid lines) to a 100mHz blend (dash lines) on RX & RY and the peaks are totally gone. This is true if I switch tfrom 01_28 to another set of blends as well, I took 100mHz as an example.
2. I've installed a set of blend filters on RX and RY called 01_28t. They are very similar to the 01_28 blends, I just moved poles and zeros around to see if it makes any difference. It does: the frequency of the peaks shifted from 0.617Hz to 0.664Hz (see plots attached).
Conclusion: We are using this 01_28 set of blends on all the units without any problem: only HAM3 seems to present this bizarre behavior... But even if I still don't understand what's happening, we have a solution. The 100mHz blends are not ideal for these DOFs, and I'll design a better set of blends ASAP.
Conclusion (bis): Current configuration on HAM3
X,Y,Z,RZ: 01_28 blend
RX,RY: 100mHz blend
Sensor correction ON on X,Y,Z.
Very nice, can you take some closed loop TFs so we can understand what is going on?
It appears that none of the MEDM screen snapshots are working - as a result we are unable to check on the vacuum status remotely.
Probably an issue with a CDS server? I can't see the Ops Schedule, LHO CDS Wiki, LHO CDS webpage.
After Kyle finished active pumping on BSC9 for today, I took V and P TFs on the main chain of ETMx. The pressure has come down to 11 Torr, and the TFs look healthy. Recall, previously this supension started rubbing around ~400 Torr so we should be in pretty good shape, finally. We'll take final TFs on Monday, but this looks very good. Attached is just the Vertical TF, but the Pitch looks well matched to the model as well, and can be found in the appropriate svncommon/Sus... directory on CDS for further weekend scrutiny if wanted. (Not committed to svn yet however.) I also looked at the reaction chain vertical DOF and it's TF looks healthy still too.
(Entry by Kyle)
Will finish roughing tomorrow and switch over to the turbo
Motivated by the rubbing saga of ETMX (15985), I have compiled a list of how much each stage of each suspension sags with temperature. This log is just like LLO 15636, except that instead of just top masses, all the stages are included here. The derivation of the temperature sensitivity is in LLO 12581.
Table 1: Vertical sag with temperature (microns / C)
Stage | QUAD | BS | HLTS | HSTS | OMC | TMS |
1 (top mass) | -106 | -38 | -37 | -62 | -50 | -88 |
2 | -182 | -57 | -60 | -96 | -64 | -129 |
3 | -223 | -58 | -60 | -96 | -- | -- |
4 (test) | -224 | -- | -- | -- | -- | -- |
These numbers were calculated using the derivation in LLO log 12581. The formula from that log is
dz/dt = -254*m*g/K [microns / C]
where m is the total suspend mass of a given SUS stage [kg], K is the total vertical stiffness supporting that mass [N/m], and g is gravity [m/s^2]. The negative sign indicates a drop in height with increasing temperature. The thermal sensitivity of the young's modulus of maraging steel is given by "The maraging-steel blades of the Virgo super attenuator." Braccini et al, Meas Sci Tecnol 11 (2000).
Table 2: Relevant SUS parameters
Spring properties | QUAD | BS | HLTS | HSTS | OMC | TMS |
Stage 1 mass (kg) | 123.32 | 40.42 | 36.46 | 8.99 | 10.02 | 123.94 |
Stage 1 stiffness (N/m) | 2889 | 2684 | 2437 | 360 | 500 | 3519 |
Stage 2 mass (kg) | 101.32 | 27.79 | 24.37 | 5.87 | 7.12 | 79.86 |
Stage 2 stiffness (N/m) | 3333 | 3540 | 2689 | 439 | 1229 | 4847 |
Stage 3 mass (kg) | 79.32 | 14.21 | 12.14 | 2.89 | -- | -- |
Stage 3 stiffness (N/m) | 4875 | 83213 | 189190 | 43139 | -- | -- |
Stage 4 mass (kg) | 39.64 | -- | -- | -- | -- | -- |
Stage 4 stiffness (N/m) | 72139 | -- | -- | -- | -- | -- |
Parameter file | quadopt_fiber | bsfmopt_glass | hltsopt_metal | hstsopt_metal | omcsopt_metal | tmtsopt_production |
Rick, Sudarshan, Shivaraj, Dave, Daniel:
late entry from Friday morning. Daniel found the PCAL binary output switching of the servo, laser power and shutter for PCAL ETMY was due to a mis-configuration within h1ecaty1. He fixed the configuration and restarted all three PLCs on h1ecaty1. He then burt restored these IOCs.
No unexpected restarts all 3 days.
Wed 7th, Thu 8th: no restarts reported
Friday 9th: no FE restarts, Beckhoff restarts as part of PCAL install at ETMY
Y1PLC1 11:42 1/9 2015
Y1PLC2 11:42 1/9 2015
Y1PLC3 11:42 1/9 2015
Alexa, Keita, Evan
Alexa and I performed our usual loss measurement: Pon = 1257(5) ct, Poff = 1304(2) ct, giving a loss of 140(16) ppm in the Y arm.
Next, we wanted to assess the amount of scatter in the arms by looking at the amount of IR light on the baffle PDs. With the arm locked to IR and the IR WFS running, we misaligned the green light and looked at the ITM/ETM baffle PDs, both with their 40 dB and 60 dB gain settings. Then we removed the IR from the arm by unlocking the modecleaner and misaligning PR2. This allows us to get the dark counts on the baffle PDs.
Relevant times are as follows, all on 2015-01-09 UTC:
We're still working on analyzing the data.
For now, here is a dtt of our measurements showing the gain settings, the IR transmission, and the power of the baffle PDs.
BPD powers are given in the following table. Dark powers have been subtracted. The data are taken from the BAFFLEPD_#_POWER channels, which already contain calibration from counts to milliwatts. Plots, data, and code attached.
BPD | Power (nW), 40 dB | Power (nW), 60 dB |
---|---|---|
ITMY 1 | 111(13) | 118(11) |
ITMY 2 | 4(12) | 10(9) |
ITMY 3 | 15(11) | 20(9) |
ITMY 4 | 110(13) | 116(10) |
ETMY 1 | 65.6(2.3) | 65.5(1.8) |
ETMY 2 | 1.1(1.3) | 1.18(24) |
ETMY 3 | 3.1(9) | 3.05(19) |
ETMY 4 | 105(13) | 104(12) |
Alexa, Kiwamu, Sheila, Koji, Evan
Finally we were able to lock DRMI with the high-bandwidth ASC loops.
The key here was to move IM4 so as to center the forward-transmitted beam on POP B. In addition to reducing the amount of offset for the INP error signals, we believe (based on camera images) that this reduced the amount of light scattered on the PR2 baffle.
After moving IM4, we then adjusted PRM and PR2 so that PRX would lock again. We then proceeded with the usual initial alignment of the corner optics.
Once DRMI had locked, we engaged the MICH, SRC1, and SRC2 loops without issue, and then transitioned them to high bandwidth (by turning off the -20 dB filters and ramping down the BS oplev damping).
Then we were able to engage the PRC1_P and PRC2_P loops without issue, and transition them to high bandwidth (by turning off the -20 dB filters, and turning on the PRM M1 and PR3 M1 locking filters).
Initially we had difficulty turning on PRC1_Y and PRC2_Y. However, we found that we could get them to work by engaging them in close succession. Kiwamu conjectures that there may be some gain heirarchy at work here.
Then we were able to engage INP1_P. Initially we put in an offset at the error point so that the loop would not immediately try to integrate away the error signal dc value. However, we were able to turn the offset off without issue.
The only tricky business here was INP1_Y. At one point (before working on the PRC loops), we turned it on (with an offset) and found that we had to flip the sign of the gain (from 300 ct/ct to -300 ct/ct) to keep the POP buildup stable. However, once we engaged it last (after all the other loops), we found that the original gain works fine. It's still unclear what's going on here.
The new slider values for IM4 are outside the "safe" range found by Keita and Alexa (LHO#). But since the IMC pointing has been changed since then, it's not clear that these safe values are still valid.
We started a (hopefully) long DRMI1f+ASC lock at 2015-01-10 05:21:00 UTC.
When DRMI locking becomes sluggish, we found it is helpful to misalign the SRM, then wait for PRMI to lock, then adjust PRM and BS to maximize POPAIR_B_RF18. Then upon breaking the lock an realigning SRM, DRMI appears to lock more quickly.
These are the calibrated error signals and the calibrated unsuppressed displacement noises for the vertex DOFs for this DRMI lock. As instructed by Kiwamu, I de-whitened the corresponding OAF channels with the filter zpk([100; 100],[1;1], 1) (gain 1 @ DC). The RMS residual motion is: MICH ~ 50 pm, PRCL < 1pm, SRCL ~ 5 pm.
Suspension clearance issues require BSC9 incursion to correct -> Aborted pump down Kyle, Gerardo -> 4 hr. vent of X-end Kyle, Gerardo, Bubba -> Removed BSC9 West door -> Installed BSC9 West door Kyle -> Decoupled purge/vent line -> X-end "Blow down" air dew point measured -11C -> Began pumping BSC9 annulus -> Expect to begin "attended" rough pumping of X-end late Saturday morning for ??? duration and finish on Sunday
Note, for clarification, Kyles alog here reports the days activities as seen through VE eyes, not an update to the pump down starting this weekend. (I first read this alog and had another heart attack until I figured out the format of his alog was a summary. The pump down abort happened this morning followed by a vent and incursion.)
With a fresh set of expert eyes on site in the form of Brett Shapiro, we embarked on evaluating ETMx in search of the pesky rubbing EQ stop. Having theoretically narrowed the possibilites down to the PUM stage of the main chain, I started by evaluating the bottom barrel stops of the PUM on the right side (SUS convention, looking from reaction chain down the arm). While the stops looked closer than the recently adjusted test mass stops (as was expected), I did not see anything glaring. I then moved to the left side of the suspension and very quickly identified the possible culprit. The bottom barrel stop on the left side nearest the reaction chain was very close to the mass, although not touching in the in-air state (as has been verified multiple times via in-air TFs), it was ~0.1-0.2mm from the mass. I also noted that the lock nuts on neither of the left side lower barrel stops were tight, in fact they were several millimeters from the EQ stop mount bar. We continued to evaluate the remaining bottom-side stops at all stages looking for other possible sources of rubbing but did not find any. As per guidance from SYS/SUS representatives, we readjusted ALL bottom-side EQ stops at the PUM, UIM, and Top Mass stages to the erring-on-the-larger-side of the 1mm spec. We also re-verified that the Test Mass and ERM stops were set to the 1.5mm spec set by Betsy earlier this week. Fingers crossed!
The above narrative focuses on the main goal of the incursion, finding the rubbing source. The step by step checklist summary is as follows:
Betsy plans to run remote TFs on this SUS over the weekend to evaluate it's health on an ongoing basis. Hopefully she will NOT call Kyle's cell!
ETMX
A lot of work has been done on HAM3-ISI for the past month. I'm trying to summarize here what we know and which path we should follow.
. We see a high Q peak around ~0.65Hz in all the local sensors (GS13+CPS) and all the DOFs, except RZ.
. This peak is present only when the Z sensor correction is ON. It doesn't matter if the Z sensor correction is coming from HEPI or the ISI, the peak is present when one of them is ON (see here and here). It doesn't seem to have any link with the X,Y sensor correction.
. The peak seems non-stationary. First of all, the peak is not the same if the sensor correction is done with the ISI or HEPI (see first attachment). Second of all, the amplitude and frequency of the peak vary with time (see second attachment and here).
. The problem doesn't come from the ground STS. We checked the electronic chain (swapping distribution chassis) and tried another ground instrument (see here). The problem is independant (see here).
. This is not mechanical/rubbing issue. We did a driven transfer function around this frequency with a perfect coherence/no sharp peak.
. The problem doesn't come from HEPI. We turned OFF the HEPI loops with no improvement (see here)
. We don't think it comes from a specific sensor. Everything looks fine wihtout sensor correction, plus the problem shows up in all the sensors (if it was , let's say, a CPS issue, it wouldn't appear in the GS13).
. It might come from the blend: by switching to a 750mHz blend, the peak seems to disappear. However, we know that the 750mHz blend is obsolete, so I wouldn't draw solid conclusions from that...
. It might come from the drive (see here)
Now it seems that the peak appears even when the sensor correction is OFF (see third attachment)! I don't know if it's a good or bad thing, but that's the first time we've seen that. The only thing I did this afternoon is switching blend filters on all DOFs and turning the Z sensor correction ON and OFF... The ground motion doesn't seem any different than usual.
. Keep investigate to see if the sensor correction is the cause of this peak
. Implementation of a slighty different blend in Z to see if it makes a difference.
. PR2 and MC2 have a mode at 0.67Hz. Could it be a weird coupling between the sensor correction and suspension? We'll try to turn the damping loops ON and OFF to see if it makes a difference
. The ISI model has been restarted before, but we haven't tried to restart the actual computer
We'll also take driven transfer functions of MC2 and PR2, to confirm their resonance Q is much lower that this feature. Further, we'll not only try an ON/OFF test of the SUS, but we'll trying *changing* the damping filters (something we want to do eventually anyways).