Evan, Sheila,
Many things were tripped this morning after the earthquake in Nepal, (ISIs, HEPIs, and suspensions). most of this went smoothly. I had trouble untripping HAM5, the screen shot I attach is the collection of screens that I was looking at when I was confused, from these I thought that the watchdog was untripped, but that was untrue, in the ISI watchdog screen the small box aroud the rouge watchdog was red. Maybe this could be more obvious, say if the watchdog indicator on the ISI overview screen was red when the rouge watchdog was tripped, and if the reset all button really reset all the watchdogs. That might have save Jeff and Jim from a flurry of saturday morning texts in the future. Jeff logged in and reset the rouge watchdog.
Also, the PSL was tripped this morning about 16 UTC (not at the same time as the earthquake), it might have been a flow sensor problem. After turning the laser on we weren't able to open the HPO external shutter from the beckhoff computer, we came to the control room and found that the flow sensor was red, we reset that but still weren't able to open the shutter from the control room. We went back into the diode room and were able to reset the shutter from there.
After the PSL came back SYS_DIAG said that the NPRO noise eater was oscillating, but we think that it is not. The attached screenshot shows 100 days of the NPRO ROO readback, this number is around -500 when the noise eater is osciallating, Each time that the laser tripps and is turnd on again this readback is high for a while before settling down. We changed the tolerance on the readback to 50 counts from 10 counts in the SYS_DIAG test.
At one point the ETMX ESD driver tripped off, and we could not reset it by toggling the hi/lo binary outputs.
We drove down to the end station, toggled the binary output, but could not hear the driver relay switching. So we unplugged the remote control dsub, activated the driver via the front panel button, and then plugged the cable back in. Toggling then worked as expected.
[Edit: this trip is happening after pretty much every lock loss. It can be untripped by bringing BO_4 low momentarily, and then toggling BO_3 as usual.]
The PSL tripped again. It was restarted in the same fashion.
Some more notes:
The PSL tripped again. I called Rick, who said it was OK to restart it.
These trips appear to be correlated with momentary trips in the diode chiller flow bit (the most recent trip is attached). The computer in the diode room shows the flow holding steady at 20 lpm or so.
These are approximately the three trip times (UTC):
The PSL tripped once more, around 2015-04-26 22:57:47 UTC. I reset it.
In order to check whether we have messed up the camera positions, I locked the interferometer and brought it to a recycling gain of 40 by steering ITMs. It seems that the ITMX camera position remained good while the ITMY camera postion was completly off for some reason. I updated the ITMY camera position.
pit: 302 -> 180
yaw: 434.3 -> 391
After Kiwamu found the good ITM alignment we tried for a few hours to reach low noise. We found the SRCL offset was breaking the lock, after the DRMI ASC turned on and as the offset was turned off. We increased the ramp time on the offset to 5 seconds, this seemed to work.
We observed the same oscillation just before full resonance that Sheila reported a couple of days ago. We tried to stay in REFL_TRANS and wait for it to subside, but this didn't work for a handful of attempts. As we were starting to motivate ourselves to investigate, the earthquake in Nepal struck.
Kiwamu, Elli
Yesterday after removing the lexan viewport covers from all of the ITM cameras, we focused the ITMX_green and ITMY_green cameras and looked at the images of the ALS beams on the ITMS when ALS is locked. We noticed that the picture on ITMY is a fairly diffuse beam with few point scatterers, while the image on ITMX is mostly point scatterers.
We wondered if these images showed a real difference in the ITMs or whether there was a difference in the cameras, for example maybe there is a smudge on the IMTY_Green camera lens which makes the beam look more diffuse than the ITMX_Green beam. So today we refocused the ITMy_Spool and ITMx_Spool cameras, which are usually focused in IR but which have no color filters on them, to green. This way we have a second pair of cameras looking at the ITMs. The saw the same difference in pattern (ITMy diffuse beam, ITMx many point scatterers) with the ITMX_Spool and ITMY_Spool cameras. This might mean there are more point scatterers around the center of ITMX than around ITMy.
We tried looking at the pixel sum of these cameras to compare the relative brightness of the scattered light. Comparing images taken with the same settings, the pixel sum is higher on ITMX_Green than for ITMY_Green (eg 1,550,000 counts on ITMY_Green vs 960,000 counts on ITMX_Green when counts on ALS-C_TRY_A_LF are 1050 and on ALS-C_TRX_A_LF 1370, with camera settings exposure=100000microsecods, analog gain=100, image type = mono12.). The IMTY_Spool camera pixel sum = 845,000 and ITMX_Spool =186,500. THis is a factor of 4 difference between the intensity on the ITM spool cameras, which we can't explain. We expected them to be roughly the same!
6.2mag quake off the Canadian coast had everything tripped and down
07:00 Cris and Karen into the LVEA
08:05 Hugh out to LVEA to fool with FF seismometer
08:25 Hugh out of LVEA
08:37 Buba to EY to check instrument air pressure transducer.
09:00 Fil to EY to assess AA/AI swap job
09:12 Bubba laeving EY. Inst Air alarm reset
11:28 Travis heading into LVEA to look for parts
11:32 Kiwamu out to help Ellie adjust cameras
11:40 Travis out of the LVEA
~12:00 Patrick restarted Beckhoff Ethercat; Rotation stages/PSL Env. etc
4:12 - no other activities to report
Scott L. Ed P. Chris S. Yesterday the crew cleaned 82 meters ending 12 meters north of HNW-4-012. Today the crew cleaned 61 meters ending 13.7 meters north of HNW-4-015. The crew also serviced the generator today.
As a part of the camera alignment study (alog 18033), I noticed that the ASC loops for green X reacted so slow that it took about several minutes to converge. See the attached screenshot. This is a time series when I turned on the ALS X green ASC loops. I was watching the ALS COMM beatnote strength at the same time as a position-sensitive sensor. I suspected that the camera loop (DOF3) is the one which is so slow, but increasing the gain of this loop told me that the UGF is already close to the 0.5 Hz mechanical resonance (10-sih dB away from instability in gain). Perhaps some kind of cross coupling between the loops exist and may be slowing them down. I am not sure how critical this is in terms of the recycling gain, but if one does not wait for these loops to settle, certainly the alignment will not be repeatable.
Just to be sure:
After the study, I restored the settings (e.g. analog gains, aperture masking and etc) of the ITMX and ITMY green cameras back to what they were yesterday. Also I restored the ALS ASC settings back to what they have been. Theoretically there is no change in the digital cameras and their associated ASC loops.
I've been trying for a while to get feedforward running on HAM's 4&5, but not having much luck. Originally, I just tried copying the filter we are using on the BSC's, only because it was there, and the test was quick and easy. And not the right answer, as (kind of) expected. In order to design a proper filter, I've taken measurements as outlined in Brian Lantz's SEI log 682. Two measurments are needed, 1. a driven tf from the FF path to the ISI GS-13's and 2. a passive tf between the St0 (or HEPI) L4C's and the St1 GS-13's. First attachment is the driven tf for Y on HAM4, second picture is of 2 passive tf's between the ISI GS-13s and St0 L4C's (in blue) and between the ISI GS-13s and the HPI L4C's (in red). The filter should be a fit of the ratio between the passive tf and the driven tf (so green / blue (OR red), depending on the path you choose). Third attachment is plot of these ratios, blue is St0 FF, red is for HPI FF. I've tried a number of filters and so far the best I've gotten is a small improvement between 20 and 25 hz, and at best, no worse any where else. Fourth attachments is of my fits to the St0 feedforward. I've tried closer fits and they didn't work either.
Just to make sure I wasn't totally missing something I tried doing the same for the BSC's and was at least able to get something that worked (fifth plot is the performance, last plot is the design). A work in progress.
There was an error on the phase part of the last plot, I grabbed the wrong data. That's fixed now.
It came up in the seismic call that there may be a sign error in either HEPI or the St0 L4C's. I think that's probably the case. I looked at the transfer function between HEPI and HAM4's St0 L4C's to the HAM5 STS. HAM4's St0 L4C's are 180 degrees out of phase from HEPI's. I checked T1000388, our design document, and it looks like our installed cartesian matrices are "right", but according to my measurements, they ain't right. This probably doesn't explain my difficulties with feedforward(maybe?), but it came up. First 2 images are are the Y direction tf from the HAM4 sensors (HEPI (red) and ISI(blue)) to the STS. X and Z were the same. I will need to repeat this sometime when I can put the ISI into a damped state so I can get the ISI's GS-13's for comparison.
The next 2 images are of the passive measurement from the previous post, but a longer, finer measurement with the phase this time. Same color scheme, red is the GS13/HEPI measurment, blue is GS13/St0 measurement.
I realigned the beam into the ISS array. The QPD signals are close to zero.
For reference, here are the DC values read by the slow signals for 3.3W input power:
PD1 | IOP-PSL0_MADC1_EPICS_CH24 | 1247 |
PD2 | IOP-PSL0_MADC1_EPICS_CH25 | 1370 |
PD3 | IOP-PSL0_MADC1_EPICS_CH26 | 1346 |
PD4 | IOP-PSL0_MADC1_EPICS_CH27 | 1487 |
PD5 | IOP-PSL0_MADC1_EPICS_CH28 | 1506 |
PD6 | IOP-PSL0_MADC1_EPICS_CH29 | 1400 |
PD7 | IOP-PSL0_MADC1_EPICS_CH30 | 1625 |
PD8 | IOP-PSL0_MADC1_EPICS_CH31 | 1504 |
I didn't check the jitter to RIN coupling as we did in the past, so the alignment might not be the optimal one yet.
In addition, it seems that PD8 has some problems, since the spectrum is clearly noisier.
J. Kissel The CAL-CS front-end model is where the infrastructure for the calibration (i.e. inverting the DARM actuation function such that a waveform calibrated into strain can be injected into the DARM loop in the correct DARM [ct] units), so I'm looking at what I need in order to fill the one filter bank that we'll use for this inversion. I've used the DARM open loop gain TF model (described in LHO aLOG 17951) to demonstrate (a) what the actuation function model looks like and (b) how simple it *could* be to invert it. In summary, we can get a ~5%, 3 [deg] frequency-dependent fidelity of the full model between 10 - 2000 [Hz] if we simply use a 1.03e-13 [m/ct] * (1 [Hz] / freq)^2 model for the magnitude, and a -1 * 176 [us] delay model for the phase. Huh! Recall that 1.03e-13 [m/ct] has a +/- 26% uncertainty because we're as of yet unable to discern the difference between optical gain fluctuations and ESD strength changes from fluctuations in charge (again, see LHO aLOG 17951 for discussion). I'll discuss with the LLO injection team to see what they'd implemented for ER6, where they've rolled off the *inverse* of this filter, and discuss the path forward, so we can get something installed by the mini-run.
Done around 12:14 to see if it might help resolve rotation stage issues.
Mike, Jeff, Jim, Dave:
Jim has built the h1hwinj1 server. This is a Scientific Linux 6.5 rack-mounted machine in the MSR with software installed as requested by the hardware injection group. Eric Thrane, Ed Daw, Peter Shawhan, Michael Landry and Jeff Kissel are commissioning this machine between now and the 5/1,2 mini run.
Attached are ASD comparing the Old (troublesome) STS2-B and the newly repaired STS2 temporarily in place in the beer garden. The current traces are from ~0130pdt--looks like a quiet time based on the PEM traces.
When I compare these to the plots (calibrations in meters) in 18007 from Wednesday AM, I observe the following:
Strong coherences are moving to lower frequencies in all DOFs.
Unity transfer function for X DOF too is moving down in frequency. The Y & Z TFs look pretty similar.
Maybe this means these signals are getting more stable. However, the Z signal of the Old & New STS2-B still don't look like the A & C units, is this an indication of the vinyl difference?
I have created a new CDS Overview MEDM screen customized more for CDS sysadmin rather than OPS. It is very similar to the CDS overview with the following differences:
The purpose of the screen is that any non-green indicator shows a possible problem which can be resolved by software means (unlike overflows which require system level intervention). The screen is called H1CDS_ENG_STATE_WORD_CUSTOM.adl and is accessible from the CDS block of the SITEMAP (ENG OVERVIEW selection).
Evan, Sheila
We have been able to lock the IFO at 10 Watts with a recycling gain of 37. We also think that we can probably improve our inital alingment scheme by adding a servo from POP A to PR3 durring the INput align step. We've closed loops around ITMX from the TRX QPDs, these were stable for almost an hour before we dropped the lock for other reasons, and helped keep the X arm build up stable as we increased the power.
Variation on Inital Alingment
At the beginning of the evening, our alignment resulted in a rather low recycling gain (15 or something) which resulted in difficulty locking. This was probably because the ITM camera references would have moved 18033. We have now gone through an initial alignment with some extra steps which brought us back to a decent recycling gain (35 before any manual adjustment or full lock ASC).
PIT (urad) | Yaw | |
PD1 | 169.2 | -311.4 |
PD4 | 134.5 | -278.9 |
mean | 151.85 | -295.15 |
This resulted in a recycling gain of about 35, so it seems like this was a sucsesful approach at least once. If there is time for some day time commissioning tomorrow, it would probably be helpful to add to the INPUT align state a loop which actuates on PR3 to center the beam on POP A. Commissioning the dither Align script for the BS to ETMY baffle PDs might also be helpful, but this is a lower priority because we aren't sure that step is necessary.
Since we have seen this week that many things (CARM offset reduction, violin and roll mode damping, and ASC error signals, possible radiation pressure instability on the ITMs) change when we go from a bad recycling gain to a good one, it seems like we will want to improve our initial alignment scheme enough to consistently bring us to a high enough recycling gain that we can use consistent settings for lock acquisition.
Avoiding oscillations durring power up
We ran into the oscillation in ITM pitch which we think is due to mis centering and radiation pressure on the ITMs several times tonight. It seems like we are more stable when the recycling gain is high, we have also slowed down the final CARM offset reduction and the power increase.
Closing ITM loops
We were able to close loops around ITMX from the QPD error signals we found last night. Here are settings:
DSOFT P (ITMX P) error signal 1.71 TRX A -1 TRX B FM offset 0.08, FM2,3,4 gain -700,000, top mass offloading on
DSOFT Y (ITMX Y) error signal 1.84 TRX A -1 TRX B FM offset -0.1, FM2,3,4,6 gain 600,000, top mass offloading on
both of these loops respond in a 2-3 seconds.
DARM OLG
Evan made a measurement of the DARM OLG, this was when we still on ETMX, DC readout with low frequency boost, and a recycling gain of 35. 11 Watts input power.
Now a large earthquake has tripped most of the seismic platforms.
While in full RF lock with a good recycling gain, turn off the green QPD servo and align the Y green beam to the arm manually (or using green WFS feeding back to PZT mirrors), then and set the QPD offsets.
Hopefully this will fix the ETMY green beam position, and makes Y arm initial alignment less convoluted.
The BS alignment slider values for PD1 and PD4 are switched in this entry. I redid the BS→EY baffle pointing today and found
P (µrad) | Y (µrad) | |
PD1 | 134.3 | −280.0 |
PD4 | 169.5 | −314.0 |
Mean | 151.9 | −297.0 |
Also, PD4 has a large dark offset (19.7 µW) compared to PD1 (−0.5 µW). The net power reported by PD4 with the PSL beam pointing onto it is 4.2 µW, and the for PD1 it is 5.2 µW. The settings for both are 0 dB gain, 0.5 A/W responsivity, and 20 kΩ transimpedance.
Evan, Sheila
We have been slowly stepping up the input power tonight, and we see the power drop when we request increased power. Here is an example, from 4-24-15 9:35 UTC. The top left panel shows the power which Evan requested, the bottom left panel shows the actual cmd position, which approriately goes in the same direction consistently when the requested power increases. The right two panels show the problem, that the readback of the actual rotation stage position goes in the wrong direction briefly when a new comand is sent. This is a real decrease in power, the lower right panel shows that the power monitor PD sees these power decreases.
If you zoom in, you can see many things.
1. At the very beginning of the rotation, the cmd position (left bottom) doesn't ramp for about 2 seconds, and suddenly it jumps up (red circle), which coincides with the reversing motion of the rotator. This behavior is repeated in every power step in Sheila's data.
(Updated after reading Daniel's comment: 2 sec delay might be a non-issue, but the jump seems to be an issue.)
2. At the end of the rotation, the cmd position looks good but the rotator jumps up, causing another power jump (blue circle). This also is repeated in every power step in Sheila's data.
3. There's one instance where the rotator reversed direction in the middle of ramping in Sheila's data (green circle).
4. There's one instance where the power increase was requested, nothing happened for 11 seconds, another power was requested, and the rotator started moving (second attachment).
(Updated after reading Daniel's comment: This might be a non-issue.)
After the power request one also needs to press the go button to get it started. So, I guess the 2s delay is just that. For the request without moving, are we sure the button was pressed?
I requested a small power step (from 2W to 2.1W) and immediately back to the original power, and the measured power dropped from 2.25W to 1.9W.
I repeated again, and the power went further down to 1.5W.
All in all, the calculated angle got back to the original number, of course, but the measured angle is down by about 0.4 degrees or so.
Daniel hit the "Go To Power" button without changing the power request, and the driver still did its thing and the power dropped.
After repeating the routine 4 times, the encoder value happened to get back to the number before Daniel started, but the power is still down from 1.5-ish to 1.1-ish W.