So what was supposed to be a simple favour for Aidan to turn the CO2y laser on turned out to be not so simple. Note: when I started this work I was unaware of the issues that Greg encountered last week (he has just put in an alog now for me alog 17302).
So at the moment the CO2y laser is still off.
Below is what observed/thought was the problem before hearing from Greg about the issues he saw previously (and is now alogging) , and also talking to Aidan about his recollections. Also will include what I think is going on for part of it and also recommendations.
Observation 1:
The rotation stage for CO2y is not working (I cant get it to do any rotation of any kind). I played with this before knowing that Greg had seen this last week also. I did not try playing with the CO2x. Based on past experience at LLO and LHO its going to be something with Beckhoff having to be reset (we really need to work out why these things just stop working). So until it is up and running I dont want to turn the laser on. And because I dont know how LHO like to proceed with doing procedures like this I took this no further. So what I need is for someone (maybe at start of next weeks maintenance period) to reset the HWP for me (probably just needs to be unplugged then replugged).
Observation 2:
I went out to the mechanical room to check the status of the power supplies etc. I immediately saw two things that peaked my interest.
1. Only one of the Kepco power supplies is on and wired up?? and the one that was on was drawing no current (see pic Kepco power supply). This I have seen before and indicates that one of the AOM drivers has faulted. But because things look to be hooked up weird (will discuss more later) looking on the tables I see that CO2y AOM driver is fine (indicated by the two green LEDs..see pic CO2yAOM driver), but the CO2x AOM driver has faulted (top LED red...see pic CO2xAOM driver). If you want to know what these fault lights mean please go to E1500133 and look at table 1 (someone should probably print this table out and stick it on or near the AOM driver for others for future reference). Now this has been seen before at LLO and the solution is to power cycle the power supply. However doing this will make the AOM start to work, and seeing as its not known when it faulted and what static diffraction setting the driver has been set to, once the reset occurs I could alter how much power is going to the interferometer (as CO2x laser looks like its being used). So again this is something should do during maintenance period (should note what power going to optic now and so when reset see how/if it changes and alter power accordingly).
2. The CO2Y laser power supply seems to be drawing a funky amount of current. 2.5Amps actually (see pic TCSy power supply), even though the laser is off. Ive even turned the key on the controller off, so it shouldn't even be in standby mode where only draws 0.7Amps. Talking to Aidan he seems to think that maybe the AOM driver/AOM is drawing current from this supply (him telling me that made me recall we saw numbers like this back in the day when we were installing and the laser was off but we had the AOM driver on). He seemed to recall the AOM driver and CO2 laser are driven off the same supply still here at LHO (which was the original design), even though the plan was to change to having the CO2 laser and AOM driver driven off separate supplies (as has been implemented at LLO...the Kepco power supplies power the AOM drivers the other supplies power the CO2 laser only). But this doesnt seem to totally match with what Im seeing in the mechanical room.
So looking at the wiring, here is what I think is the current setup at LHO. We are in some funky mid change between having one power supply do both laser and AOM driver and having separate supplies for all. In more detail......The two shelves showing the power supplies for TCS are shown in TCSracklowershelf and TCSrackuppershelf. How it should be as per the latest plan that I know of (and is at LLO) is that the on the lower shelf should hold the power supplies that power one table (ie say Y ) and for the upper shelf the power supplies for the other table (X) with seperate supplies for the lasers and the AOM drivers. What seems to be the case is that:
*on the lower shelf we have the Instek supply (I think thats the brand name....its the the one with the digital readout reading its drawing 2.5Amps that says TCSY on top of it) powering BOTH the CO2y laser and the CO2y AOM driver (I think this because the laser is OFF, as is the controller, but the AOM driver on the Y table has not faulted and is "running".The x table AOM driver has faulted and is not "running" and 2.5amps is an approximate number that the AOM driver could be drawing).
* The Kepco power supply on the lower shelf is wired up but not drawing any current at the moemnt and thus and looks to be powering the AOM driver for the X table (why... because the x table AOM driver has faulted its not drawing any current). It should be wired to be controlling the Y table if supply in the position at..or at the very least labeled clearly as to what it is powering. Mind you I could be wrong and its powering something else
* The instek supply (the one showing the digital readout of 22.42 Amps) on the upper shelf is powering ONLY the CO2 x laser (its drawing this current as the laser is on).
*The Kepco power supply on this upper shelf is not even wired up and is thus off.
What I suggest is at some stage when have some time (maybe during maintenance period next week), is to wire the system as per the new design (and as how is at LLO). One Kepco power supply is used for the AOM driver on one table and the other Kepco power supply be wired to the AOM driver on the other table. And then just have the other power supplies individually controlling the CO2 lasers. Should then also have the power supplies for one table on one shelf and the power supplies for the other table on the other shelf.
Also when turn laser back on need to see if still only drawing half the current like Greg saw and if so investigate (probably cable connection is the problem like in the past). SO quite a few things would lile to do at next weeks maintenance period.
10 days ago, Gabriele and Evan reported that there seemed to be a large offset in the IMC locking loop (alog 17131). In order to investigate this issue, I started from checking the health of the IOT2L optical table -- specifically we wanted to confirm that there is no stupid clipping or something crazy.
I did not find a clipping or anything crazy on the table.
(Health check of IOT2L)
I went to the IOT2L and checked the REFL path using a IR viewer and card, but I did not find an obvious clipping or any suspicious things. I found an iris which was letting the main beam go through but also scattering ghosts beams. The iris was in front of the BS which steers the beam to the REFL RFPD. I took out the iris and re-adjusted the razer blade beam dumps which have been meant to damp those ghosts. Also, I checked the centering of the beam on the REFL RFPD by steering the BS, but it was already centered.
(What happened to MC trans digital camera ?)
BTW, the IMC trans camera had been displaying nothing these days according to the hourly archives probably since some time in February. I realigned the camera as it seemed to be misaligned in YAW. After adjusting the exposure, I realized that the beam shape completely changed from the image that I remembered -- see the attached pictures. The first attachment is the one from Feb-04-2015 and the second from today after my realignment. During the realignment process, I moved the camera position to see if this image is something bogus or real. I am concluding that this image is real. Something must have happened in the IMC trans path between early February and now. Sad.
Hugh and Krishna had found the BRS software had crashed pretty much on schedule, so Sheila and I went to the end-station to restart it. Pretty straightforward. Alogs 13817 and 15005 will get you everything you need to do it. The BRS is rung up right now because we went out to check the mass damper positions, but it should settle down in 30 minutes or so.
K. Venkateswara
I took a look at the BRS_RY_IN channel and it looks like BRS started to get damped but then the damping failed (see attached pic). It is currently oscillating with 500 count ampitude. I'm not sure why this happened but it currently looks like the mean position of the damping turn-table has changed by 45 degrees causing it to neither damp nor drive BRS. A repositioning of the turn table might help. If not, I'd suggest turning off the damper temporarily.
Looks like Jim and Sheila did everything right, but the damper is a bit finicky.
Looks like it did damp down eventually. I have no idea what happened, but it is possible that the damper turn-table got stuck, moved to a new mean location and then somehow got stuck again and returned to a reasonable location. In any case, looks like BRS is damped now and working well.
LVEA: Laser Hazard Observation Bit: Commissioning 07:00 Karen & Cris – Cleaning in the LVEA 08:22 Hugh – LVEA for HEPI maintenance work 08:36 Jodi – In the LVEA for 3IFO stuff 08:37 Christina – Forkliftng boxes from LSB to VPW 08:45 Jim B. – Starting work on the NTP server move 08:45 Matt – Turning on CO2-Y laser 08:50 Richard – Vacuum survey in the LVEA 08:51 HFD on site testing fire alarms 08:55 Kyle & Gerardo – Swapping out Ion pumps on HAM1 & HAM2 09:00 Peter, Jason & Ed – In the PSL for RefCav and FSS work 09:01 Corey – Working in the squeezer bay 09:34 Richard – Going to End-Y with HFD to test fire alarms 09:51 Hugh – Finished with HEPI in LVEA – Working on pump controller. 09:52 Jodi – Out of the LVEA 10:00 Praxair delivering N2 to Mid-X 10:03 Paradise Water on site 10:30 Filiberto – Working on TCS-X RF cabling 10:48 Richard – Back from End-Y 10:55 Kiwamu – Checking IOT2L for beam clipping 11:00 Doug – Looking for OpLev parts near HAM2 11:00 Peter, Jason, & Ed – Finished in the PSL 11:05 Reset HEPI watchdog counters 11:38 Dave – Restarting new PEM corner station model & making cable changes in CER 11:42 Kyle & Gerardo – Out of the LVEA 11:50 Dave – DAQ restart 11:50 Kiwamu – Out of the LVEA 12:35 Karen – Going to End-Y 12:35 Cris – Going to End-X 12:43 HFD on site testing smoke alarms at End-X 13:10 Kyle & Gerardo – Leak checking at HAM1 & HAM2 13:51 Karen – Leaving End-Y 13:52 Filiberto – Connecting TSC RF cables 13:55 Kiwamu – Checking mirror pointing in IOT2L 14:22 Filiberto – Out of the LVEA 14:30 Kiwamu – Out of the LVEA 14:41 Jason – In LVEA getting parts from the OpLev cabinet 14:56 Jason - Out of the LVEA
We have turned off 3 LVEA heaters today as the winter season departs. It appears that temperatures in the LVEA are now back to nominal after an excursion of ~0.5F.
12 hours shown on the plot.
We'll continue to monitor the response.
I collected my performance data with the BSC in the right blend configuration this morning (sans modified blend filter, from my alog earlier today) and updated the HAM spectra from Friday, with slightly better color choices, I hope. See attached pdfs.
This is an issue from last week. While doing chiller maintenance last week I noticed that the power supply for the TCSx laser was running at only half power. I checked the cables and didn't find any obviously loose connections, but this has been an issue in the past. Less likely, but still possible is the RF driver failing. While on the table I noticed the rotation stage has once again gone glitchy and won't respond to requests or return to home. Most likely requires a Beckhoff reset. There is and has been a beam block in the laser path. For now the TCSy laser is disabled by the MEDM gate.
Actually the TCSy is disabled by turning the key off also at the control box (I didnt know about these issues at the time when I started trying to turn the CO2y laser on today (alog to come)).
Also like Greg mentions I would suspect the cable connection if is onlydrawing half current as this has been a known issue on this table before see alog 16495.
P. King, J. Oberling, E. Merilh
Summary
FSS RefCav TPD started drifting again, went in to re-align. Check mirrors M23, M22, and M21, all found loose. M23 and M21 pitch adjustments recovered RefCav TPD, M22 had no effect; all pitch adjustment screws locked. FSS RefCav TPD left at 1.56V. Adjusted M36 to optimize power incident on PMC_TRANS PD, now reading 23.03 W (up from 22.99 W). Measured FSS TF, UGF measured at 195 kHz.
Details
The FSS RefCav TPD had begun to drift down again, so we went in and checked the remaining 3 mirrors in the FSS beam path: M21, M22, and M23. We first checked M23 and found the pitch adjustment screw to be loose. Turning this screw CW (pitching the mirror down) caused the RefCav TPD to change from 1.23V to 1.53V; the adjustment screw was locked. M22 was also found unlocked, but did not effect the RefCav Trans; this adjustment screw was also locked. The final mirror in the FSS path that we had not inspected before was M21, and lo-and-behold, the pitch adjustment screw was unlocked. Turning this screw CW (pitching the mirror down) took the RefCav TPD from 1.53V to 1.56V; this adjustment screw was then locked. We will continue to monitor the FSS RefCav TPD.
Since the transmission through M23 is responsible for the PMC_TRANS reading, we slightly tweaked M36 to optimize the reading of the PMC_TRANS PD. Before the adjustment the PMC_TRANS was reading 22.99W, it is now reading 23.03W.
Finally, we took a TF of the full FSS loop, and measured the UGF to be ~195 kHz. Peter has the full TF data and will add it as a comment to this post.
Checked and set all LVEA Supply Accumulators to 70psi; Return Accumulators set to 20-30psi.
Issues found at H(am)5R(eturn), H6S(upply), H5S, H4S, B(SC)3E(ast)S,B3W(est)S,B3WR,B2N(orth)S(unable to charge due to interference), B2NR, B1NS, B1SR, H3S, H1S, H1R, H2R.
Wow, maybe easier to list the ones that did not need some gas. Most of the Supply units that needed charging where at 20psi or below. This should make a good improvement in pump pressure fluctuation coupling into to platform motion. I expect next check should see a much shorter list.
Found that someone had decided to store unused 2" Angle Cable Tray Support Assemblies under BSC1. Sliding these in, the Feet on the Assembly was just the right height to hit the valve knob and turn it. Fortunately, the drain exit is capped and no leak occurred. Please mind the caution tape and be aware of the pushing and pulling things in and around the low dark places where HEPI resides.
Noticed that the Vacuum Pipe, Instrument Air Piping and Conduit are all contacting the SR3 OpLev Transmitter Pier near its base. This is just East of HAM4 on the North Side.
Reset the watchdog counters for HAM2, HAM3, HAM5, ITMX, and ITMY
I got a chance to test the modified blend filter that JeffK and I came up with on Friday(alog 17256). It looks like it most does what we expected. Attached plots show X, Y and Z dofs for both stages., dashed lines are the before, solid lines are after, red is St1, blue is St2. I don't show it, but there was no change in performance below 1 hz. I think we might be able to push the elliptical filter down a little more, get some improvements closer to 5hz, but that will have to wait while I run down other projects.
I've installed this on ETMY, ITMY and ITMX, but it's only running on the Y arm chambers, for now.
A side note, the blend filter banks for the BSC's are getting a little weedy. It's probably getting to be time to do a little clean up and make things more consistent between chambers.
Dan, Kiwamu,
At one point last night, we noticed that a roll mode rang up at ~14 Hz. The peak height in the DARM spectrum (ffted with a 0.1 Hz BW) exhibited a anomalously high value of more than 10-12 m/sqrt Hz.
So we worked on damping the peak, this time with the AS_WFS_A_45Q signal (alog 16864) which is a state-of-the-art Livingston technique. We were successfully able to damp the mode to an ordinary height of about 10-14 m/sqrt Hz level in the DARM spectrum.
(Identification process)
First of all, see how terrible the roll mode was:
The red curve is from Mar-17-2015 8:51:00 UTC and we were intentionally stayed on the rf DARM rather than the DC readout to combat the roll mode. Even though there was some confusion in identifying which suspension had been rang up, eventually we confirmed that the roll mode was from ITMX by taking coherence between AS_WFS_A_45Q and the OPLEV_PIT signal of each test mass. In fact, the ITMX oplev was able to see the roll mode in its spectrum with a peak ~ 10 times higher than the noise floor while the rest of the oplevs did not see a visible peak.
In addition, after the successful damping on ITMX, the AS_WFS_A still showed a bit high peak but with a slightly lower frequency (~ 200 mHz ) than that of ITMX. We then identified that this was in turn from ETMY by looking at the coherence again. This lead us to another damping work on ETMY which we were also able to damp.
(Damping setup)
The AS WFS_A signal was routed from the ASC model with a gain of +1 (in ASC_OUTMATRIX_TESTMASS_DAMP). I found that both were damped well with a positive gain in SUS-E(I)TMX(Y)_M0_DAMP_R.
As for ITMX, I ended up with FM3 (-100 dB), FM4 (bp13.9) and a gain of +40. Although I had to start from a gain of +1 in order not to saturate the DAC due ot the high amplitude in the roll mode. Then I gradually increased the gain as the DAC counts reduced. Once I went to a gain of more than 10, the mode was damped relatively quickly probably on a time scale of about a couple of minutes.
As for ETMY, I ended up with FM4 (bp13.9) and a gain of +0.0008. At one point I went to a gain of +0.005, but this actually started increasing the peak height. So I had to set the gain back to + 0.0008. This loop could damp out the mode on a time scale of a couple minutes as well.
(Why they rang up ?)
We don't know. One observation is that before the modes rang up, Sheila was trying to increase the recycling gain by manually touching the ITMs.
What's the exact frequency (or at least to ~0.01 [Hz] precision)? Remember, I was able to distinguish two mode frequencies in LHO aLOG 16868: a 13.18 [Hz] mode or the 13.81 [Hz] mode, which I'd narrowed down to an ITM and an ETM mode respectively. It would be good to catalog is the ITMX mode you say rung up. @DetChar -- you're more than welcome to answer this, if Dan or Kiwmau can't get back to it quickly. The time you should use is listed in the bottom corner of the DTT screen capture.
since the color of the Time text is the same as the reference, I guess that's not the right time. Better to use the time of the RED live trace.
Looks like about 13.98 Hz, accurate to 0.01 Hz. I used 5 mins of data, 120 sec FFT, overlap of 0.9. I used the time mentioned in the log entry (8:45 UTC Mar 17) since the data didn't look right around the time in the DTT screen. We don't have a nice tool for quickly marking the peaks in spectra. I'll look into making one.
Seismic: Hugh doing HEPI maintenance in corner station Jim performance testing on ETMX, & Filter work on ETMY Reboot of End-X BRS computer 3IFO: Jodi moving and labeling 3IFO items in the LVEA Bubba hooking up 3IFO Quad LS and BS storage boxes to N2 system Corey moving bins in and out of the LVEA PSL: Work on RefCav alignment & TFs on FSS CDS: Model and DAQ restarts Jim moving the NTP server VAC/FMP: Swapping ion pumps at HAM1 & HAM2 Tumble weed bailing Hanford FD on site testing fire alarms
Dan, Kiwamu, Sheila
Today we made several gaurdian changes:
I looked for a signal from the QPDs which was insensitive to TMS angle. For the X arm pitch I used TRX A - 0.5 TRB and for Y arm pitch 1TRY_A-0.65 TRY_B seemed good. I closed a verry low bandwidth loop to ITMX using this signal, screen shot is attached.
We found that we could increase the recycling gain from ~27 to ~33 by moving the ITMs with ASC lops closed on IM4, PR2, BS, common ETMs, and differential ETMs. We had to move ITMX by as much as 5 urad to do this, and we found that we would loose the lock soon after getting a good recycling gain. Kiwamu calcualted that by moving ITMX 5 urad, we are moving the spot on the test masses by 1 cm assuming that the ETM is controlled. This may be due to a problem with the DHARD loop.
We also spent some time looking for a Transmon QPD signal that would be insensitive to motion of the transmon. We closed a loop to ITMX pitch using TRX_A - 0.5 TRX_B, the attached screenshot shows the settings. This loop was verry low bandwidth.
We had a second incident of a permanently ramping slider today, earlier we saw this with SR2 Yaw and just now (3:56:20 UTC) we saw it again with SR2 pitch. The offset had ramped to the correct value (probably in the 10 second ramp time), but the ramping indicator stayed on. This wouldn't be a problem except that we are using .is_ramping in the guardians, and they get confused by this situation.
Again I changed the ramp time, and Dan moved the slider one click and moved it back again, and the ramping indicator went off.
Dave investigated this earlier in the day when we saw it with yaw.
More details on the scattering Dan mentioned on Friday, with some new and re-interpreted details (the responsible motion is horizontal) that became clear after further investigation.
This last Monday, DARM spectra showed a double scattering shelf occasionally reaching 60 Hz and 120 Hz (Figure 1) and even higher. I searched for the source of the scattering path length variation by looking for motion sensors that detected maximum motion at the times that the higher frequency scattering shelf reached its highest frequency. The top trace of Figure 2 is a 700s time series of DARM, band passed between 90 and 145 Hz. Each of the spikes in the time series was produced when the scattering shelf reached into this band. The lower plot shows that the maxima in one of the OMC OSEM signals coincides with the spikes in the DARM plot above it. All OMC OSEMs show this correlation except the side sensor (on the short side). I found no other GS13 or OSEM channels that showed this correlation; most importantly, it was not in the GS13 signals from HAM6. Figure 3 is a zoom in to one of the clusters of scattering spikes, showing that the scattering spikes appear to occur at the steepest part of the OSEM signals (when velocity would be highest), and that the time spacing between DARM spikes agreed with the resonant frequencies of the OMCS. Beating between 2 of the suspension modes seems to cause the variation in motion.
Using the 1 um/count calibration of the OSEM OUT channels, I obtained average velocity spetra that, for some OSEMs, reached 10 um/s (Figure 4).
In order to produce a shelf out to 60 Hz, the rate of path length change for a single bounce would have to reach about 30 um/s. The OSEMs measure displacement of the top mass, M1, and the OMC hangs below it. At the resonant frequencies of this suspension, the motion of the OMC, while damped, would still be greater than the top mass. Also, the 10 um/s figure is only an average rms. Thus the OMC motion can account for the 60Hz shelf with a single reflection.
There are two shelves in Figure 1 spaced by a factor of 2 in frequency. The spectrogram in Figure 5 shows that the frequency spacing of the two shelves is always a factor of 2. The shelf at 120 Hz in figure 1 is about an order of magnitude below the shelf at 60 Hz. If this represents a scattered beam that reflects twice instead of once, then the reflectivities at each of the two extra surfaces would have to be very high. While there may be other mechanisms to generate the double shelf, it is probably worth looking for bright beam spots on highly reflective surfaces in HAM6.
Finally, it would be nice to reduce the motion of the OMC by a factor of ten, particularly, the side to side and rolling motion perpendicular to and about the line connecting the two suspension points of M1, motion detected by the LF, RT, T2 and T3 OSEMs.
It's likely the OMC ASC control (Kiwamu, Daniel, Keita)
Summary:
OMC BOSEMs are usually very quiet, but they show extremely big motion that Robert showed only when the IFO is in lock with DC. In-lock VS out-of-lock ratio is huge at about 3 orders of magnitude.
It turns out that this comes from OMC ASC control actuating on the OMC suspension.
OMC YAW has a large coupling to the distance between the OMC and the IFO because the rotation axis is fairly distant from the first steering mirror on the OMC breadboard, probably 18cm or so, and therefore is likely this is the main motion coupling to the scattering path modulation.
Apart from identifying the scattering source, there could be some mitigation tasks that we could do.
We will test the two TT mitigation tomorrow.
Details:
Attached shows the same OMC BOSEM velocity signals that Robert used (the only difference is that this is calibrated in um/sec, not m/sec).
Solid lines are now, broken lines are when Robert took his measurement. At around big peaks, there's 3 orders of magnitude difference. It turns out that they're mostly like solid lines show, and becomes excited only when in DC-lock.
Kiwamu will fill in more details.
I attatch a 24-hrs trend of some relevant channels from Mar-10-2015. As shown in the trend, it seems that every time the OMC-ASC loops are in action actuating on the OMC suspension, the OMC OSEMs read high fluctuation as weel as a big shift in the DC values. When the OMC ASC loops are not in action, the OSEM readouts are quiet.
On Friday we reduced this scattering noise below the usual noise floor by reducing the OMC ASC gain by ~10x. This reduced the UGF of the QPD loops down to 0.1Hz. This is around the UGF of the dither loops, which explains why we only saw this noise recently, when the QPD loops were used in low-noise. See the attached plot of the OMC SUS longitudinal signal: the blue traces are the QPD loops in a high-gain state, red is low-gain. The purple traces are from March 4 when the OMC was aligned with the dither loops 9 (the dither signals are rolled off above 2Hz since the signal-to-noise above that frequency is not good). The black traces are the quiescent OMC SUS noise without ASC feedback. For the current noise floor between 10-100Hz, the motion of the red and purple traces are low enough to keep the scattering from being the limiting noise source.
That said, this scattering knowledge means that the experiment of feeding back alignment signals to the OMC SUS should end. I've added OM3 to the ASC model, so we can feed back the DC centering signal from AS_B to OM1-3, along with the two degrees of freedom from the OMC. This will give us a 3x3 control matrix for the HAM6 alignment, similar to what's being done at L1. It ignores the centering on AS_A, but centering on AS_B should be sufficient.
Btw the dither loops need to be re-commissioned because I moved the dither lines up to ~1.7kHz. This changed the sensing matrix, it needs to be remeasured and inverted for a new control matrix. Tomorrow we will 1) switch the control topology to use the OMs rather than the OMC SUS, and 2) switch the sensing topology back to the dither for low-noise operations.