J. Kissel, S. Dwyer Sheila's taken another DARM open loop gain transfer function just a second ago at my request, such that we can get a handle on how much the loop (and therefore the calibration) has changed since Koji improved the DCPD compensation (see LHO aLOG 17647. I'll process and make a statement tomorrow but I record some details here for the record. Parameters needed for Kiwamu's bare-bones calibration model: DARM Gain = +1300 DARM FMs ON = 1,2,3,4,5,6,7,9 ETMY UIM Gain = +0.2 UIM FMs = 1,2,3,5 The .xml has been copied over and committed to the calibration repository, here /ligo/svncommon/CalSVN/aligocalibration/trunk/Runs/PreER7/H1/Measurements/DARMOLGTFs/ 2015-04-06_H1_DARM_OLGTF_LHOaLOG17710.xml
Today I started hacking through the weeds that had grown up the SUS SDFs for the last 2 unattended weeks. While I still have more pruning to do, I addressed:
- ODC DAMP GOOD state updates on medm and SDF where DIFFs were found - likely more tidy up needed per sus.
- Low noise BIO/COIL changes 17504 and 17478.
- Cleaned up SUS TEST banks (ignoring GAIN, OFFSET, TRAMP values, but continuing to watch SW1 and SW2 for when bank gets left on)
- Found lots of LIMITs engaged, so wrote their values into the safe.snap
- Set SDF to ignore SW2 of many LOCK and TEST banks which were in SDF DIFF state since now SUS GRD aligns and misaligns via these, alog 17259. Reenabled SW1 monitoring in many cases where inputs will remain enabled now. Where alignment offsets were written into TEST banks, I've written the values into the sus snap.safe.
- Zeroed out the R0 P TEST OFFSET of 200 that Keita put in a long time ago and inadvertently got written into the ETMx safe.snap and subsequently reenabled. These should be off, 16865 - Today I turned the R0 P offset off, again. Also, the R0 Y TEST bank offset (0.0) was enabled so I turned that off. Our understanding is that no R0 chain offsets should be enabled.
One more thing to highlight from this effort: all settings in the OMC and OMs in HAM6 have been captured, and there are no longer any differences with the table. Thus, this will be a reference point to get back to after the vent and/or we can monitor what has been done while in chamber such that we understand all that has changed during the vent. Note, I attach the lists of channels that are currently *not* monitored by the SDF system so we're aware of what *won't* be tracked.
8:04 Jeff B. to LVEA checking on HAM6 vent prep
8:18 Jeff B. out
9:28 Kyle to EX collecting stuff for HAM6
9:46 Fil to H2 PSL area for dewpoint meter cabling
9:50 Bubba to EX
9:54 Jeff B. don moving dessicant cabinet to high bay
10:14 Bubba back
10:15 Kyle back
10:41 Richard and Fil to EE bay, Fil to H2 PSL area
10:48 Jeff B. to LVEA checking dust monitor at HAM6
11:12 Richard and Fil back
12:34 Elli to HAM1 checking out equipment
12:43 Elli back
13:14 Kyle to HAM6
13:41 Elli to LVEA
13:46 Hugh locking HAM6 HEPI
14:54 Sudarshan to LVEA
15:00 Daniel to EY
15:10 Hugh back
15:27 Suresh to LVEA adjusting HAM3 OpLev
15:34 Ed to LVEA assisting Suresh
16:00 Ed and Suresh back
Started ~2pm. Switch ISI to Damped. Locked HEPI carefully while Isolation loops were on, for a while. Still some shifts though:
X -5000nm, Y +5500nm, Z +12500nm, RX +5000nrads, RY -500nrads, RZ +400nrads. ISI returned to High-Isolated.
This time on St1, only in Z. Good improvement at a few hz, not as broad as the St2 improvement. First attched plot shows the improvement (mostly between 1 and 5 hz), dashed lines are off, solid are with the ellipse on, red and blue are ground, greens are St1, purples are St2. Second plot shows the change to the CPS part of the blend, red is stock, blue is with the ellipse. I am only trying Z right now because the sensor correction is offloaded to HEPI, so changing the CPS phase won't affect the SC performance. It might be possible to do something similar in X and Y, but harder, because changing the CPS plant phase will affect the sensor correction.
J. Kissel, J. Warner One might notice that there appears to be little improvement on ST2, so one might ask "why bother?" Here's why: On ST2, the platform was already close to being limited by sensor noise, but judging by a comparison of the shape of the ASD between ST1 and ST2, ST2 was still dominated by residual seismic noise from ST1 at these frequencies. The improvement of a factor of 50 at 3 [Hz] pushes the residual ST1 motion *well* below the sensor noise floor. Even though this doesn't improve the displacement of ST2 by much, it should improve the stationarity of ST2 significantly, given that sensor noise is stationary and residual ST1 motion from ground is not. Another victory for Jim!
Laser Status: SysStat is good Front End power is 31.8W (should be around 30 W) FRONTEND WATCH is RED HPO WATCH is RED PMC: It has been locked 5 day, 5 hr 33 minutes (should be days/weeks) Reflected power is 2.0 Watts and PowerSum = 24.6 Watts. (Reflected Power should be <= 10% of PowerSum) FSS: It has been locked for 0 h and 1 min (should be days/weeks) TPD[V] = 1.443V (min 0.9V) ISS: The diffracted power is around 9.6% (should be 5-9%) Last saturation event was 1 d, 23 h and 6 minutes ago (should be days/weeks)
Per JeffK's request, I have switched ETMX to the "windy" configuration. The ISI was already running the 90mhz blend since sometime this weekend. I've now also engaged the low frequency sensor correction on the X direction with the BRS turned on. I've been fiddling a little with the Z blend filters on St1, adding low pass filters to the 90mhz Z blend, but it doesn't seem to affect the St2 performance much. I'll post more on that in a bit.
Our goal is to prove that not only can this X DOF "windy" configuration (90 mHz blend, broadband, low-frequency sensor correction, with the BRS) replicate the performance of ISI in its "nominal" X DOF configuration (45 mHz blend, narrow-band 0.43 [Hz] sensor correction, no BRS), but it can do *better* than the nominal configuration during high winds.
As such, we want to compare three different configurations:
(1) Nominal ("45 mHz blend, derosa 0.43 Hz only sensor correction, no BRS") [we should already have plenty of this data]
(2) Windy when BRS doesn't work ("90 mHz blend, derosa 0.43 Hz only sensor correction, no BRS") [we may already have plenty of this data]
(3) Windy with a functional BRS ("90 mHz blend, mittleman broadband low-frequency sensor correction, BRS on") [what we'll get now, hopefully if the winds pick back up]
We've got some more of configuration (2) this morning,
2015-04-06 16:00 UTC to 17:00 UTC -- winds were max ~25 [mph]
so that will be great fodder to compare what Jim's put the ETM ISI into now ( configuration (3) ).
And for the record, it was switched into this configuration (3) at 2015-04-06 21:10 UTC. Now we just need the winds to pick up again...
Ask and ye shall receive! Just after Jim switched to configuration (3) described above, the winds picked back up to > 20 [mph] for an hour or so -- see attached screenshot (note that I've plotted the mean minute trend over 24 hours; I would plot the max as well, but my remote trending abilities are limited). Thus, a good time to use in the above described comparison for config (3) is from 2015-04-06 21:30 UTC to about 23:30 UTC, plenty of time for several averages of a low frequency ASD.
Richard, Jim, Dave:
we removed the 6th ADC from h1seih23 after it was consistently showing a 4000 count offset in its channel-13 (14th channel). The entire ADC-ribbon-interfacecard were replaced with off the shelf spares which fixed h1seih23.
We installed the suspect ADC-ribbon-interface on the DTS x1seih23 front end as the 3rd ADC (ADC2) on Friday afternoon (3rd April). We restarted the front end several times that afternoon, but did not see any offset in any ADC channel. We left this running over the weekend.
This morning the ADC channels still looked good. I restarted the IOP model 5 times, no problems were observered. I started a series of power down tests, and found that the IOP model would not run (with very large irig-b errors) if I powered down the IO Chassis, using the front panel switch, for only 5 seconds. If I powered the IO Chassis down for 1 minutes (either switch only or switch and DC power cord disconnect) the IOP would start correctly.
I then performed the power cycle test 5 times, no re-occurance of ADC offsets were observed.
Bottom line, we dont know why this ADC's ch13 went into a 4k offset mode at 12:30am Friday. We cannot reproduce the error.
In preparation for tomorrow's HAM6 vent
Due to an issue currently under investigation by Jamie, the Guardian nodes for BS, ITMX, ETMX, and ETMY were restarted at ~10:30am PST. The SPM diff monitor was reporting various SWSTATs were being toggled quickly (a few times per second), but the SUS MEDM screens reported no such behavior.
This seems to just be a problem with the reporting of the status of monitored channels. This shouldn't have any adverse affect on the actual operation of the nodes. And it seems to have been a transient problem, since the issue went away after the nodes were restarted. Still investigating.
Prep for HAM 6 vent ongoing
Jim W. working on filters for ISIs
HAM 6 SUS TFs have been taken in prep for vent (look good), SDF work ongoing
CDS installing dewpoint monitor cables in H2 output vicinity
Purge air being turned on
2nd cleaning on cleanroom complete
OpLev working on replacement lasers, PR3 needs recentering
PLL work today
Jeff B. moving dessicant cabinet into high bay for craning tomorrow
PCal work Tues. at EY, maybe EX
Koji, Sheila
Koji and I searched around for the source of the periodic (every 4 second) glitch in DARM control when locked on ALS figure attached to 17684. The problem seems to be the DIFF PLL board, we were able to see the glitches in the PLL control signal when it was locked to a marconi instead of the DIFF beatnote. We locked the DIFF beatnote using the COMM PLL board, saw no glitches, and also swapped the ADC cables to make sure the DIFF ADC channels were fine. I don't know where the spare is, so we are elaving the chassis in the rack for now, but the board needs to be swapped before we can get back to locking the full IFO reliably.
Just too be clear, it seems likely these are a different class of glitches from those described in 17576 and linked alogs
We made some additional tests this morning but couldn't see anything obviously wrong. There is a fair amount of RF interference between the 3 VCOs which depending on the exact frequencies looks bad.
Some additional observations:
Eventually we figured out that with DIFF and COMM unlocked, these 4-second glitches were showing up in the Y end green control signal, and not in X.
So we drove down to EY and found that the EY ESD driver was tripped. We could hear that it was trying to reset itself every 4 seconds or so. Untripping the driver made the glitches go away.
It is unclear whether this was actually kicking the test mass, or coupling into the green readout some other way.
[Ed. Merilh, Jason Oberling, Doug Cook, Suresh D]
Doug replaced the diode laser at the HAM3 oplev this morning after it was fixed for reducing glitching (SL No. 197) in the optics lab 2. We wanted to let it settle for a while and reach thermal equilitbrium before adjustting the power level. We did that around 12:30PM and I checked the results around 4PM. The laser is still settling down as seen in the attached plots. We plan to monitor it for another day.
Sl. No. 197 Diode laser requires readjustment of power.
Please see the two attached plots.
1) The first shows short term trends of the laser power as obtained from the HAM3_OPLEV_SUM_IN1_DQ channel. The first panel shows the RIN spectrum. Note the two orders of increase in when we go below 1Hz towards 0.1Hz. This indicates power instability at low frequencies (A signature of glitching). The second panel of this attachment shows hte time trend of this signal which shows gradual increase in glitch rate after the first hour or so. The laser is moving from a low glitch rate to a high glitch rate power level due to thermal changes. The third panel shows the same info in greater graphic detail with time evolution of the spectrum (spectrogram)
2) The second attachment is a long term 1s trend of the same SUM signal. It shows that the initial power setting was okay and had few glitches if any. However the power dropped over the following half a day and moved to an unstable zone. It stabilised there and continued to glitch because it has landed at the edge of a stable zone and is now mode hopping.
Cure: Increase power from 47900 SUM counts to 48500 counts. Further one day of observation to see if it has worked.
The diode laser Sl No. 197 which was under observation at HAM3 oplev has been performing well for the past six hours. There were a few minor glitches after Tuesday morning maintenance started. This could have been some heavy stuff moving around on the floor and disturbing the HAM3 Oplev Transciever. The behaviour over six hours has been summarised in the attached plots of amplitude spectrum, time series and spectrogram.
Today has been a rough day for the input HAM chambers. This morning we found that a channel went dead in expansion chassis (see Richards alog for the resolution), and troubleshooting involved a lot of restarts of SEIH23. This afternoon, after that problem was sorted, Evan found that the SEI MEDM screens for HAM2&3 were frozen. Dave and JimB should be posting a log about the resolution of that issue.
It would be very good if Detchar could at do some comparisons of the H3 IPS on HAM3 HEPI with other sensors to see if the failure that took out this chamber this morning (killing commissioning efforts until ~now) gave us any warning. I attach some dataviewer trends of the IPS blend ins, and I can kind of convince myself that the horizontal loops all look a little noisier 18 hrs ago versus now. Not necessarily true, I haven't looked in any detail and I'm just guessing based on the max/min being noisier 18 hours ago.
Plots like this, for the last couple of weeks. This is a spectra from 23 hours ago before the trip at 01:00 local today. Maybe also look at impacts on the ISI and/or MC2 PR2?
I've attached spectra of H2 and H3 spaced by three hours. There's a clear indication that H3 is going bad on Apr 2 between 18 and 21 UTC. The next spectra pin the start time to between 19:20 and 19:25 UTC (I think that's noon local time). The time series, which starts at 19:15, shows that the problem may come in the form of bursts of noise. Update: I've added an hour-long time series of the channel, high-passed at 10 Hz. The problems start 12 minutes in. It looks like there are bursts of noise, as well as maybe an increase in the overall noise level.
According to the summary pages, this had an impact on the ISI and on the optic motion. The ISI spectrogram and optic motion spectrogram show an increase in noise right around 19:20 UTC. If we need a monitor for this sort of problem, a BLRMS from 30 to 100 Hz would probably work. The sensor is just flat white noise there, until the signal goes bad. Attached is a 12 hour BLRMS showing the onset of the problem.
Sadly, this doesn't show up in the BLRMS signals that we currently have on HEPI. The HPI-HAM3_BLRMS_X_30_100 channel is the BLRMS of the L4C cartesian signal in the 30 to 100 hz band and it doesn't show the sensor going bad. Attached plot is the same 12 hour window that Andy plots just above, and the problem is not apparent. Mo channels, mo problems.
Jim, Krishna,
After the troubles described in alog 17386, BRS was turned off to allow it to damp down naturally. This morning Jim and I restarted the BRS as attempted earlier in 17309. This time, the amplitude of the balance was low enough and we simply had to reposition the damper and start the software. Everything worked as designed. The attached plot shows the damping of the beam-balance.
After things had settled down I checked the spectra of the signals coming out of BRS and the tilt-subtracted super-sensor and everything looks normal. Wind speeds were at 0-10 mph and the subtraction looked good as seen in the attached pdf. The first plot shows the ground T240 X motion, the BRS_RY ( * w^2/g), ST1 T240 X and the super-sensor. The next plot shows coherences between these sensors.
Better late than never, right?
While I was at EX on this day, I also went out to the chamber and checked the corner 3 CPS rack (re: the bumbling line, most recently discussed in alog 17681) and found nothing amiss. All cables are secured, all CPS field racks are grounded and have all their screws. Haven't had time to look at this in more detail (pulling cables, cards, turning stuff off and on again, etc.).
Here are some notes on the recent DARM calibration for future reference.
(A) We intentionally do not switch the simulated ETM actuators in the CAL-CS model when we switch to ETMY for low noise in the DARM control.
Even though we switch the actuator from ETMX to ETMY for the LSC DARM control to achieve low noise, the calibration filters for the actuators in CAL-CS remain using the *ETMX simulated actuator path*. Since we adjusted the ETMY ESD digital gain such that the response of ETMX and ETMY ESDs are the same with a precision of 10 % in frequency band from 20 to 60 Hz, the calibrated spectrum after the ETMY transition should be still valid with a precision of 10 %. In other words, we forced the actual ETMY ESD to be identical to that of ETMX so that we don't have to manipulate the filters in CAL-CS. In fact, when we transition from ETMX to ETMY, we do not see visible reduction or increment in the DARM spectrum at low frequencies below 100 Hz, which convinced us that the calibration remained consistent. Then, once we decrease the ETMX ESD bias all the way to zero, we see noise reduction in 20-100 Hz band due to lower ESD noise.
The adjustment of the ETMY ESD digital gain was done on the last Tuesday, Mar-24th (alog 17411). The following is some detail of what I did for the adjustment. First, I measured the transfer function from ETMX_L3_LOCK_L_IN1 to LSC-DARM_IN1 in 20-60 Hz band by swept sine while the interferometer stayed fully locked on DC readout. Then I measured the same transfer function but with the ETMY ESD driven. So the transfer function I took was from ETMY_L3_LOCK_L_IN1 to LSC-DARM_IN1. At this point, I noticed that the ETMY ESD response was weaker than that of ETMX by an overall scale factor of 1.25. Note that both ESDs use the linarization. So I increased the ETMY ESD gain by the same factor in ETMY_L3_LOCK_L_GAIN. So this gain is now 1.25 and this is already coded in the ISC_LOCK guardian. Then in order to check how good the matching is between ETMX and ETMY, I ran another swept sine on ETMY with the new gain. However this still gave me a bit too-weaker ETMY ESD by 10% (I attach the dtt xml file). Since we knew that the overall scaling factor of our calibration is already uncertain by the same level (on the order of 10%, see for example alog 17082), I did not try to get better matching between them. We will revisit the ETMY calibration at some point.
(B) The DARM spectrum below 2 Hz is not trustable.
There are two reasons.
By how much is the calibration off at low frequencies because we do not include the low-frequency control filters, i.e. Kiwamu's point (B)2? See attached. This is the (driven) transfer function from the DARM_CTRL input of the CAL CS model (actually H1:CAL-CS_DARM_FE_ETMX_L3_ISCINF_L_EXC) to the output of the actuation function chain (actually H1:CAL-CS_DARM_DELTAL_CTRL_WHITEN_IN1). At these low frequencies, the formulation of DELTA L EXTERNAL is entirely dominated by the actuation function times the control signal, A * DARM_CTRL. So, any discrepancies between what we believe is the most accurate reconstruction of A and the real A is by how much that calibration is off. To give a few frequency points in words, at 0.1 [Hz] the discrepancy is roughly an order of magnitude too low, and at 0.02 [Hz] is almost two orders of magnitude off. In other words, any plots of DELTA L EXTERNAL is *under* estimating the displacement by ~10 at 0.1 [Hz] and by 100 at 0.02 [Hz]. This transfer function comparison also reveals (maybe?) what problems that Kiwamu mentions in (B)2. One can see *without* the low frequency compensation, the TFs clean. *With* the low frequency filters engages, the transfer function becomes ratty both at low and high frequencies. Maybe this is a problem with numerical precision in the filter computation?