We have had SDF examples of a setting change which is smaller than the displayed resolution and so hidden by rounding. For example, I have changed a gain setting from its safe.snap value of 1.0 to 0.9999999999999998 and the SDF difference table shows both values being rounded to 1.000000. To get the actual value to 15 decimal places, use caget with the "-e 15" argument (without this argument it rounds to 1.0)
caget H1:PEM-MY_CHAN_17_GAIN
H1:PEM-MY_CHAN_17_GAIN 1
caget -e 15 H1:PEM-MY_CHAN_17_GAIN
H1:PEM-MY_CHAN_17_GAIN 9.999999999999998e-01
The snapshot of the table shows the two values are apparently the same when rounded to 6 decimal places.
6:45 Jeff - LVEA West Bay SUS Storage, LVEA East/West Bay Laser Safe 7:15 Cris – LVEA in 7:26 Mitch – LVEA in 8:12 Andres – LVEA West Bay w/Jeff 8:35 Mitch – LVEA out 8:45 Kyle, Gerardo – LVEA N. Crane 9:15 Filberto – To End X 9:20 Kyle, Gerardo – LVEA out 9:30 Doug, Jason – To both ends 9:39 Christina – To both Mids for storage 9:57 Joe – End X 10:01 Corey – To End Y 10:14 David, Jeff – ETMX, TMSX, ETMY, TMSY brought down to Safe for restart 10:21 Truck in to take water samples 10:27 Cris – End X 10:29 Doug, Jason back from ends, to LVEA 10:45 Water delivery 11:00 Jeff, Andres – Out of LVEA, back to Full Laser Hazard 11:03 Doug, Jason out of LVEA 11:05 Kyle, Gerardo – Out of LVEA, they left a cart on next to BSC7 11:17 Sudarshan, Darkhan – To Ends to set up Pcal Lasers 11:39 Corey – Back from Ends 13:23 Kyle – Into LVEA to turn on power supply 13:52 Sudarshan, Darkhan - Done at Ends
J. Kissel, K. Izumi, A. Staley, S. Dwyer ETMX, ETMY, and HAMs 2 and 3 were involved in maintenance day, and recovery was little-to-no different than any other Tuesday. Chasing alignment in the IMC and ARMs have been the most time consuming parts, though my impression is that it has been a little bit shorter / better today and the SUS are not nearly as far off as they have been. Indeed, in order to hard-reboot the SEI HAM23 front end and IO chassis, we needed to take down the IMC, all its associated suspensions (and the IM and PR suspensions), and the HAM 2-3 ISIs and HEPIs -- and this DID NOT shake the MC REFL path enough to cause any misalignment in pitch as we've seen in the past (see Integration Issue 854. Rather than going through my usual detail play-by-play, I've decided that its more productive to list what we should change as a result of today's experiences in order to keep getting better at the recovery. Lessons (at least *I*) learned from today, in no particular order: - We need to have guardian transition the BSC ISI Stage 2 Gains from HI to LO when a SEI manager requests to go from FULLY_ISOLATED to OFFLINE for ALL BSCs, because requesting to go from WATCHDOG TRIPPED (afer computer reboot) to OFFLINE to FULLY_ISOLATED after does *not* transition the gains to LO before getting started. - We need to have the HAM ISI guardians *ensure* the GS13 gains are HI. - We need to have an IMC "down" state that is actually DOWN, (or perhaps call it "OFFLINE") -- specifically the IMC WFS are turned off so no angular control is fed to the MC suspensions, and (less importantly) no longitudinal control is fed to MC2. - The new(ish) IMC ASC DOF4 / MC WFS center servo really can pull the MC REFL pointing off into the weeds, so these should be the first thing turned off when ramping down the IMC. - The transition from ISOLATED to OFFLINE on the HAM's watchdog trips the ISI halfway through the turn off process more often than not, and its likely because the GS13 gains remain in high-gain mode indefinitely. - We need a guardian to manage the HSSS (today it was the IMs, but another day it was the RMs, etc.). - We need to calculated / measure / install / save alignment offset calibrations for the HSSS. - We need a standardized, well-ordered check-list procedure for taking chambers / parts of the offline -- and then bringing them BACK online. - We need to either resurrect / commission / trust the SUS DRIFT_MONITOR, or find another tool that we can quickly assess how all SUS alignments have changed from prior to the start of a maintenance day to after finishing. - We need a tool that takes the output of the above (i.e. the assessment of the alignment), and *fixes* any SUS that are out of alignment. - When the global "initial" alignment changes -- even by a few microradians here or there -- the optical gain of the IFO changes, which means we need to re-adjust ISC parameters. - We need to include turning ON and OFF the ESD Bias into the SUS ETM guardians, such that it is ON in every state but SAFE. Further, we should pick a standard location to turn ON and OFF the ESD BIAS path. We're not sure if we're *fully* recovered at this point, but we have gotten as far as a short DRMI locks with ARMs off-resonance, and we push onward. Regarding the Guardian Logs: SEI_ETMX Manager Guardian was requested to go to OFFLINE from FULLY_ISOLATED between 2015-01-20 18:15:24.753 UTC 2015-01-20 18:16:43.581 UTC
Rana, JeffK, Kiwamu,
We did some maintenance items on IMC ASC. The major activities are:
Also, tomorrow or sometime this week, we might insert a lens in the REFL gigE path as the REFL beam now is a bit too big to be able to cover the spatial higher order modes.
Some issues with the current implementation:
Tonight we have twice had MC lock losses where MC1+MC3 got completely misalinged, although they didn't trip. The second one of these was at the same time as a lockloss of the ALS, but we don't know if the IMC caused it or ALS. This was at 6:35 UTC, Jan 21st.
We made the decision not to increase the spot size. This was done at LLO to decrease the jitter coupling into the ISS. The in-vacuum ISS doesn't require this.
The IMC seem to have worked much more reliably, before we added DOF4. Please turn it off for good.
J. Kissel, J. Warner, D. Barker, E. Merilh, TJ Shaffer The one final quiver in our arrow against the sharp HAM3 0.6 [Hz] resonant feature was a computer (as suggested e.g. here), so we did so this morning. Sadly, it did not cure the problem. For now, we continue on, running in the configuration defined by Seb last week, see LHO aLOG 16100, in which the RY blend filters at a higher blend configuration, which is the only thing we can find that seems to help (see LHO aLOG 16001). This has now officially become a low-priority until it begins to quantifiably impact IFO performance. A Recap of all of the things we've tried that DIDN'T make the feature go away: - Restarting the front-end process LHO aLOG 15759 - Full power-down of Front-end and I/O Chassis (This aLOG) - Coil Driver Swap LHO aLOG 16071, LHO aLOG 16061, LHO aLOG 15981 - Changing Suspension Damping LHO aLOG 16013, LHO aLOG 16001 - Changing Isolation Loops LHO aLOG 16001 - Location of Sensor Correction; ISI vs HPI LHO aLOG 15941. LHO aLOG 15933 - Changing from STS2 B to A; In analog LHO aLOG 15811, In digital LHO aLOG 15927, LHO aLOG 15894 - Checking between ADC and sensor correction bank LHO aLOG 15783 - Turning OFF CPS satellite amplifiers, checking jumper settings (oscillators, jumpers, etc.) LHO aLOG 15751 - Adding a large mechanical offset to relieve supposed mechanical rubbing LHO aLOG 15751 - Waiting LHO aLOG 15565
I have entered an integration issue for this at https://services.ligo-wa.caltech.edu/integrationissues/show_bug.cgi?id=1005
Sudarshan, Darkhan
Pcal Lasers at ENDX and ENDY are switched on but blocked at the transmitter module so that the light does not get into the test mass. These lasers will remain on until we report the change of status.
Andres R., & Jeff B. We craned the 3IFO Beam-Splitter storage container and the four (4) Quad Lower Storage containers (for Q6, Q7, Q8, & Q9) over the H1 Input arm and placed them into their final long-term storage positions. These containers are ready to be connected to the N2 system.
Bubba G., Andres R., Mitch R., Jeff B. This morning Mitch and Jeff finished loading the Elliptical Baffles into HAM ISI Storage Container #2. Jeff added a 4" witness wafer. After securing the top on the container it was craned over the H2 input arm and placed into its final lone-term storage position. We next craned the 3IFO BSC #4 container into its final long-term storage position. This container was the last of the suspensions long-term storage containers to be filled and stowed. The cleanroom and dust monitor (#10) supporting this effort have been powered down.
Pulled ETMX photodiode amplifier chassis from EX. Keita reported voltage outputs seemed to be reversed (negative output). Found the wiring for the outputs of the amplifier box wired wrong. Corrected wiring and placed unit back in rack. Patrick had to restart the Beckhoff computer for EX. D1301017 SN S1400075
ETMX Baffle PDs now output positive voltage.
Gains were set to 0dB instead of 60.
Nominal currents for these (H1:AOS-ETMX_BAFFLEPD_1_NOMINAL and H1:AOS-ETMX_BAFFLEPD_4_NOMINAL) were set to 0.1mA instead of 0.
"powerOk" for ITMX in ditherAlign.py was set to 0.2 instead of 0.4.
After these changes the ditherAlign.py works for ITMX.
Kyle, Gerardo Craned pump cart out of Beer Garden with West crane -> Pump cart left running (near BSC7, west side of XBM)
Kyle ~1330 hrs. local -> Energized controller -> Decoupled pump cart from pump port -> New ion pump is OK
Filiberto, Patrick Filiberto told me that the Beckhoff wasn't running after he took out a chassis at end X to switch some wires. I took the easy approach and just restarted the computer. I burtrestored the channels back to 8:10 this morning.
The XEND has been under vacuum now for 9 days since the last test mass adjustment. Plot attached.
model restarts logged for Mon 19/Jan/2015
2015_01_19 10:40 h1fw0
2015_01_19 12:07 h1fw0
2015_01_19 13:46 h1fw0
2015_01_19 15:37 h1fw0
2015_01_19 15:41 h1fw0
2015_01_19 16:06 h1fw0
2015_01_19 23:06 h1fw0
all unexpected restarts. Conlog frequently changing channels report attached.
[Koji Dan]
Summary
The estimated displacement noise of the OMC cavity due to the actuator noise is shown in Attachment 1
Motivation
There was a suspicion that the OMC PZT driver noise was limiting the OMC cavity length noise. (LHO ALOG 16089). In the previous measurement, the output monitor ports for the HV and LV PZT drive were used. However, it was highly likely that the readout noise levels of these monitor ports are not low enough to measure quiscent noise levels of the PZT drivers. Therefore we wanted to try direct measurements of the LV and HV PZT driver noises at the output of the PZT drivers.
Method
First of all, the HV power supply of the OMC PZT driver was turned down from 100V to 10V in oder to insure our safety. Of couse, this is not an ideal condition in terms of the proper noise measurement. However this should not be a problem, in principle, as the noise of the final HV stage should be limited by the OP27 at the input stage of this section. (The HV amp section of this board has cascaded amps in a single feedback circuit.)
A DB25 breakout board was inserted between the driver rack unit and the output cable to the vacuum feedthrough.
Pomona grabber clips were attached at the pins across each PZT electrodes. The voltage was fed to an AC coupled SR560 with a gain of 100 and HPF with fc at 3Hz (6dB/Oct).
After the measurement, the breakout board was removed, the cable was restored, and the HV power supply was reverted from 10V to 100V.
HV driver results
Here the result includes the explanation including the measurement of the monitor outputs.
1. Evaluation of the AC/DC mon outputs (Attachment 2)
The raw output noise levels of the AC/DC monitor were compared with the readout circuit noise model by LISO. Basically this plot indicates that the outputs are limited by the noise of the readout circuit above 100Hz. Below 100Hz, it looks the measure noise levels are above the modeled noise levels. The difference is too small to declare the measure noise level is the true indication of the PZT noise.
2. Direct measurement (Attachment 3)
The noise levels across the HV PZT (green) was shown in the figure. Here AC/DC Mon measured voltage noise levels were converted to the equvalent output voltage for comparison.
The noise level of this measurement (indicated by a dark green dotted curve) was 4nV/rtHz down to 10Hz, which was as indicated in the SR560 spec. So this noise level is quite reliable. We can declare that the measurement below 100Hz indicates the actual voltage noise across the PZT. Otherwise, the minumum of all three measurement at each frequency above 100Hz should be taken as an upper limit.
The black curve in the plot is the modeled noise by LISO. The measued noise was consistently higher than the model. The reason of the descripancy is not known.
In the first plot, the noise level from the direct measurement was plotted below 100Hz after converting it to the displacement of the cavity.
LV driver results
Here the result includes the explanation including the measurement of the monitor outputs.
1. Evaluation of the AC/DC mon outputs (Attachment 4)
The raw output noise levels of the AC/DC monitor were compared with the readout circuit noise model by LISO. The circuit noise dominates the outputs except for the AC Mon between 2kHz and 20lHz.
2. Direct measurement (Attachment 5)
The direct measurement indicates better noise level than the result of the AC monitor in all frequency. The direct measurement has clear gap from the measurement noise level below 100Hz and above 2kHz. So the measurement is reliable in these bands, and otherwise the level is an upper limit.
The black curve in the plot is the modeled noise by LISO. The measued noise was consistently higher than the model. The reason of the descripancy is not known. 20KHz.
In the first plot, the noise level from the direct measurement was plotted below 100Hz after converting it to the displacement of the cavity.
It is not clear to me that reducing the HV from 100 to 10V will yield a reliable HV output noise measurement. Certainly, the closed loop gain of the OP27 feedback loop will change. Whether that produces a measurable and meaningful change in the HV output noise spectrum is unsure to me. I tend to prefer a careful capacitive coupled measurement on this, but of course that requires a different skill set from the people doing the measurement to ensure the test equipment and the personnel hazards are properly mitigated.
It would have been better to measure the noise with the proper supply voltage. But we did not have that option in the weekend.
We are tracking the ADC errors which are being seen on the STATE_WORD for the IOP models. These are a feature RCG2.9 They are raised very infrequently, work to remove them is ongoing.
I have cleared the errors by pressing the DIAG RESET buttons on the IOP models. Here is the maximum ADC0 hold time for the models which were showing the error:
h1iopsusb123: adcHoldTimeEverMax=23, adcHoldTimeEverMaxWhen=1105450127 (Jan 16 2015 13:28:31 UTC)
h1iopsush34: adcHoldTimeEverMax=25, adcHoldTimeEverMaxWhen=1105533748 (Jan 17 2015 12:42:12 UTC)
h1iopsush56: adcHoldTimeEverMax=24, adcHoldTimeEverMaxWhen=1105583419 (Jan 18 2015 02:30:03 UTC)
h1iopsusey: adcHoldTimeEverMax=21, adcHoldTimeEverMaxWhen=1105219714 (Jan 13 2015 21:28:18 UTC)
h1iopslc0: adcHoldTimeEverMax=22, adcHoldTimeEverMaxWhen=1105241179 (Jan 14 2015 03:26:03 UTC)
See more complete L1 data at aLOG entry 16455. We need a bit more proc file data to characterize the limits
We have had several incidents where MC1+MC3 trip, and sometimes also cause HAM2 ISI to trip. Kiwamu found that there were large signals coming through DOF4 (16128), although there was some other underlying problem which was causing MC2 to get large length signals.
We Had DOF 4 off for most of the day saturday, once we found and fixed the PSL noise eater oscillation we turned it back on. This morning we dropped the mode cleaner lock when we were attempting to lock the X arm in IR, which also tripped MC1, MC3, and HAM2 ISI (this lockloss is the kind of thing that is expected to happen once in a while, and should not cause a cascade of trips). We have again held the offsets on DOF 4, because somehow this loop makes the sysem more fragile. Since holding the outputs we have had several mode cleaner lock losses without any trips.
Perhaps it is worth while going through the WFS alignment process once. The most likely scenario is that the WFS_B_DC_PIT and YAW offsets were turned on while the spots were centered. I am sure the following steps are obvious but let me put them down here any way for future reference:
1) Start with a good alignment of the IMC input beam. Off load any offsets from the suspensions. Maximise IMC output by hand if necessary.
2) Unlock the mode cleaner by invoking the "Down" state in the IMC Guardian
3) Using the MC2 Alignment sliders misalign the MC2 by a milliradian or so in pitch till the IMC stops flashing. (Use StripTool)
4) Open the WFS DC screens and inside that open the WFS_A and WFS B_PIT and YAW filter banks. Switch off any offsets that may be present.
5) Using the IMC WFS picomotors center the direct reflection spots on the WFS A and WFS B.
6) Switch off the DOF4 inputs in YAW and PIT. Make sure that the history is cleared and that all servo outputs are zeroed.
(At this stage we have ensured that the gaussian sidebands (24MHz) are centered on the WFS sensors)
Now we start setting up the DOF4 servos:
7) Bring the MC2 Back into alignment. Note that there is a wait time of several minutes before the top stage of the MC2 moves into place. There is a long time constant low pass filter on the alignments sliders
8) Due to some hysteresis somewhere, you may need to search for good high intensity flashes. They will be within a few tens of microrads from the previous setting. (StripTool)
9) Bring the IMC back to "LOCKED" state in IMC Guardian. Keep the DOF4 loops off at this stage.
10) As the mode cleaner alignment loops (DOF1,2,3) work the reflected field is extinguished and some wierd pattern develops in the IMC REFL camera. This will take about 3 to 5 mins or so.
11) This wierd shape causes a spurious offset to appear on the WFS_A and WFS_B DC signals.
12) Zero these signals using the offsets in the WFS_DC PIT and YAW filter banks.
13) If everything is stable switch on the DOF4 filter bank inputs.
14) Keep track of the DOF4 outputs. They should not be monotonously building up to values larger than a few tens. If they cross 100 then something is misaligned.
15) Currently DOF4 uses WFS_B. WFS_A serves as an out of loop sensor. Trend the WFS_A PIT and YAW after a day and see if there is any drastic change. It indicates a drifting alignment causing an evolving IMC REFL pattern. Not great news but not a deal breaker either. As long as it is not monotonously running away to values outside [-0.25, +0.25] it should be okay.